<?xml version='1.0' encoding='utf-8'?> version="1.0" encoding="UTF-8"?>
<!DOCTYPE rfc [
  <!ENTITY nbsp    "&#160;">
  <!ENTITY zwsp   "&#8203;">
  <!ENTITY nbhy   "&#8209;">
  <!ENTITY wj     "&#8288;">
]>
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>

<rfc xmlns:xi="http://www.w3.org/2001/XInclude" submissionType="IETF" category="info" consensus="true" docName="draft-ietf-bmwg-ngfw-performance-15" number="9411" ipr="trust200902" obsoletes="3511" updates="" submissionType="IETF" xml:lang="en" tocInclude="true" tocDepth="3" symRefs="true" sortRefs="true" version="3">

  <!-- xml2rfc v2v3 conversion 3.15.1 -->
  <front>
    <title abbrev="Benchmarking Network Security Devices">Benchmarking
    Methodology for Network Security Device Performance</title>
    <seriesInfo name="Internet-Draft" value="draft-ietf-bmwg-ngfw-performance-15"/> name="RFC" value="9411"/>
    <author fullname="Balamuhunthan Balarajah" initials="B" surname="Balarajah">
      <organization/>
      <address>
        <postal>
          <street/>
          <city>Berlin</city>
          <code/>
          <region/>
          <country>Germany</country>
        </postal>
        <phone/>
        <email>bm.balarajah@gmail.com</email>
      </address>
    </author>
    <author fullname="Carsten Rossenhoevel" initials="C" surname="Rossenhoevel">
      <organization>EANTC AG</organization>
      <address>
        <postal>
          <street>Salzufer 14</street>
          <city>Berlin</city>
          <code>10587</code>
          <region/>
          <country>Germany</country>
        </postal>
        <phone/>
        <email>cross@eantc.de</email>
      </address>
    </author>
    <author fullname="Brian Monkman" initials="B" surname="Monkman">
      <organization>NetSecOPEN</organization>
      <address>
        <postal>
          <street>417 Independence Court</street>
          <city>Mechanicsburg</city>
          <code>17050</code>
          <region>PA</region>
          <country>USA</country>
          <country>United States of America</country>
        </postal>
        <phone/>
        <email>bmonkman@netsecopen.org</email>
      </address>
    </author>
    <date day="22" month="10" year="2022"/>
    <area/>
    <workgroup>Benchmarking Methodology Working Group</workgroup> month="March" year="2023"/>
    <area>ops</area>
    <workgroup>bmwg</workgroup>
    <keyword>NGFW</keyword>
    <keyword>NGIPS</keyword>
    <keyword>benchmarking</keyword>
    <keyword>performance testing</keyword>
    <keyword>security testing</keyword>
    <abstract>
      <t>This document provides benchmarking terminology and methodology for
      next-generation network security devices devices, including next-generation
      firewalls (NGFW) (NGFWs) and next-generation intrusion prevention systems
      (NGIPS).
      (NGIPSs). The main areas covered in this document are test terminology,
      test configuration parameters, and benchmarking methodology for NGFW NGFWs and
      NGIPS.
      NGIPSs. (It is assumed that readers have a working knowledge of these
      devices and the security functionality they contain.) This document aims
      to improve the applicability, reproducibility, and transparency of
      benchmarks and to align the test methodology with today's increasingly
      complex layer 7 security-centric network application use cases. As a
      result, this document makes RFC3511 RFC 3511 obsolete.</t>
    </abstract>
  </front>
  <middle>
    <section numbered="true" toc="default">
      <name>Introduction</name>
      <t>18
      <t>It has been 18 years have passed since the IETF initially recommended test
      methodology and terminology for firewalls (<xref <xref target="RFC3511" format="default"/>). format="default"/>.
      Firewalls have evolved significantly from the days of simple ACL access control list (ACL)
      filters. As the underlying technology progresses and improves,
      recommending test methodology and terminology for firewalls,
      requirements, and expectations for network security elements has
      increased tremendously. Security function implementations have evolved
      and diversified into intrusion detection and prevention, threat
      management, analysis of encrypted traffic, and more. In an industry of
      growing importance, well-defined and reproducible key performance
      indicators (KPIs) are increasingly needed to enable fair and reasonable
      comparison
      comparisons of network security functions. These reasons led to the
      creation of a new next-generation network security device benchmarking
      document, which makes <xref target="RFC3511" format="default"/> obsolete. Measurement The measurement of
      performance for processing of IP fragmented IP-fragmented traffic (see Section 5.9 of
      <xref target="RFC3511" format="default"/>) was sectionFormat="of" section="5.9"/>)is not included in this document since IP
      fragmentation does today not commonly occur in traffic anymore, unlike how
      it might have been at the time when <xref target="RFC3511" format="default"/> was
      written. It should also be noted that <xref target="RFC2647" format="default"/> retains
      significant value and has been was consulted frequently while creating this
      document.</t>
      <t>For a more detailed explanation of what an NGFW is is, see the Wikipedia
      article <xref target="Wiki-NGFW" format="default"/>.</t>
    </section>
    <section numbered="true" toc="default">
      <name>Requirements</name>
      <t>The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
      "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY",
      <name>Requirements Language</name>
        <t>
    The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL
    NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
    "<bcp14>MAY</bcp14>", and
      "OPTIONAL" "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as
    described in BCP 14 BCP&nbsp;14 <xref target="RFC2119" format="default"/>, target="RFC2119"/> <xref target="RFC8174" format="default"/> target="RFC8174"/>
    when, and only when, they appear in all capitals, as shown here.</t> here.
        </t>
    </section>
    <section numbered="true" toc="default">
      <name>Scope</name>
      <t>This document provides testing terminology and testing methodology
      for modern and next-generation network security devices that are
      configured in Active ("Inline", see Figures <xref target="figure1" format="default"/> format="counter"/> and <xref target="figure2" format="default"/>) format="counter"/>) mode. It covers the validation of security
      effectiveness configurations of network security devices, followed by
      performance benchmark testing. This document focuses on advanced,
      realistic, and reproducible testing methods. Additionally, it describes
      testbed environments, test tool requirements, and test result
      formats.</t>
      <t>The performance testing methodology described in this document is not
      intended for security devices/systems devices or systems that rely on machine learning or
      behavioral analysis. If such features are present in a Device Under
      Test/System
      Test / System Under Test (DUT/SUT), they should be disabled.</t>
    </section>
    <section anchor="Test_Setup" numbered="true" toc="default">
      <name>Test Setup</name>
      <t>The test setup defined in this document applies to all benchmarking
      tests described in <xref target="Benchmarking" format="default"/>. The
      test setup MUST <bcp14>MUST</bcp14> be contained within an Isolated Test Environment isolated test environment (see
      Section 3 of
      <xref target="RFC6815" format="default"/>).</t> sectionFormat="of" section="3"/>).</t>
      <section anchor="Testbed_Configuration" numbered="true" toc="default">
        <name>Testbed Configuration</name>
        <t>Testbed configuration MUST <bcp14>MUST</bcp14> ensure that any performance implications
        that are discovered during the benchmark testing aren't due to the
        inherent physical network limitations limitations, such as the number of physical
        links and forwarding performance capabilities (throughput and latency)
        of the network devices in the testbed. For this reason, this document
        recommends avoiding external devices devices, such as switches and routers routers, in
        the testbed wherever possible.</t>
        <t>In some deployment scenarios, the network security devices
        (DUT/SUT) are connected to routers and switches, which will reduce the
        number of entries in MAC (Media Access Control) or ARP/ND (Address Address Resolution Protocol/ Protocol /
        Neighbor Discovery) Discovery (ARP/ND) tables of the DUT/SUT. If MAC or ARP/ND tables
        have many entries, this may impact the actual DUT/SUT performance due
        to MAC and ARP/ND table lookup processes. This document also
        recommends using test equipment with the capability of emulating layer
        3 routing functionality instead of adding external routers in the
        testbed.</t>
        <t>The testbed setup for Option 1 (<xref target="figure1" format="default"/>) is the
        RECOMMENDED
        <bcp14>RECOMMENDED</bcp14> testbed setup for the benchmarking test.</t>
        <figure anchor="figure1">
          <name>Testbed Setup - Option 1</name>
          <artwork name="" type="" align="left" alt=""><![CDATA[+-----------------------+ alt=""><![CDATA[
+-----------------------+                   +-----------------------+
| +-------------------+ |   +-----------+   | +-------------------+ |
| | Emulated Router(s)| |   |           |   | | Emulated Router(s)| |
| |    (Optional)     | +----- DUT/SUT  +-----+    (Optional)     | |
| +-------------------+ |   |           |   | +-------------------+ |
| +-------------------+ |   +-----------+   | +-------------------+ |
| |     Clients       | |                   | |      Servers      | |
| +-------------------+ |                   | +-------------------+ |
|                       |                   |                       |
|   Test Equipment      |                   |   Test Equipment      |
+-----------------------+                   +-----------------------+]]></artwork>
        </figure>
        <t>If the test equipment used is not capable of emulating OSI layer 3
        routing functionality or if the number of used ports is mismatched
        between the test equipment and the DUT/SUT (need (which is needed for test equipment
        port aggregation), the test setup can be configured as shown in <xref target="figure2" format="default"/>.</t>
        <figure anchor="figure2">
          <name>Testbed Setup - Option 2</name>
          <artwork name="" type="" align="left" alt=""><![CDATA[
 +-------------------+      +-----------+      +--------------------+
 |Aggregation Switch/|      |           |      | Aggregation Switch/|
 | Router            +------+  DUT/SUT  +------+ Router             |
 |                   |      |           |      |                    |
 +----------+--------+      +-----------+      +--------+-----------+
            |                                           |
            |                                           |
+-----------+-----------+                   +-----------+-----------+
|                       |                   |                       |
| +-------------------+ |                   | +-------------------+ |
| | Emulated Router(s)| |                   | | Emulated Router(s)| |
| |     (Optional)    | |                   | |     (Optional)    | |
| +-------------------+ |                   | +-------------------+ |
| +-------------------+ |                   | +-------------------+ |
| |      Clients      | |                   | |      Servers      | |
| +-------------------+ |                   | +-------------------+ |
|                       |                   |                       |
|    Test Equipment     |                   |    Test Equipment     |
+-----------------------+                   +-----------------------+]]></artwork>
        </figure>
      </section>
      <section anchor="DUT-SUT_Configuration" numbered="true" toc="default">
        <name>DUT/SUT Configuration</name>
        <t>The same DUT/SUT configuration MUST <bcp14>MUST</bcp14> be used for all benchmarking
        tests described in <xref target="Benchmarking" format="default"/>. Since each DUT/SUT
        will have its own unique configuration, users MUST <bcp14>MUST</bcp14> configure their
        devices with the same parameters and security features that would be
        used in the actual deployment of the device or a typical deployment.
        The DUT/SUT MUST <bcp14>MUST</bcp14> be configured in "Inline" mode so that the traffic is
        actively inspected by the DUT/SUT.</t>
        <t><xref
        <t>Tables <xref target="NGFW_Security_Features" format="default"/> format="counter"/> and <xref target="NGIPS_Security_Features" format="default"/> format="counter"/> below describe the RECOMMENDED <bcp14>RECOMMENDED</bcp14> and
        OPTIONAL
        <bcp14>OPTIONAL</bcp14> sets of network security features for NGFW NGFWs and NGIPS, NGIPSs,
        respectively. If the recommended security features are not enabled in
        the DUT/SUT for any reason, the reason MUST <bcp14>MUST</bcp14> be reported with the
        benchmarking test results. For example, one reason for not enabling
        the anti-virus feature in an NGFW may be that this security feature was
        not required for a particular customer deployment scenario. It MUST <bcp14>MUST</bcp14> be
        also noted in the benchmarking test report that not enabling the
        specific recommended security features may impact the performance of
        the DUT/SUT. The selected security features MUST <bcp14>MUST</bcp14> be consistently
        enabled on the DUT/SUT for all benchmarking tests described in <xref target="Benchmarking" format="default"/>.</t>
        <t>To improve repeatability, a summary of the DUT/SUT configuration configuration,
        including a description of all enabled DUT/SUT features MUST features, <bcp14>MUST</bcp14> be
        published with the benchmarking results.</t>
        <t>The following table provides a brief description of the security
        features and
        feature; these are approximate taxonomies of features commonly
        found in currently deployed NGFW NGFWs and NGIDS. NGIPSs. The features provided by
        specific implementations may be named differently and not necessarily
        have configuration settings that align with the taxonomy.</t>
        <table anchor="Security_Feature_Description" align="center">
          <name>Security Feature Description</name>
          <thead>
            <tr>
              <th align="left">DUT/SUT Features</th>
              <th align="left">Description</th>
            </tr>
          </thead>
          <tbody>
            <tr>
              <td align="left">TLS Inspection</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT intercepts and decrypts inbound HTTPS traffic between
          servers and clients. Once the content inspection has been completed,
          the DUT/SUT encrypts the HTTPS traffic with ciphers and keys used by the
          clients and servers. For TLS1.3, TLS 1.3, the DUT works as a middlebox
          (proxy) and it holds the certificates and Pre-Shared Keys (PSK) (PSKs) that
          are trusted by the client and represent the identity of the real
          server.</td>
            </tr>
            <tr>
              <td align="left">IDS/IPS</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects and blocks exploits targeting known and
	  unknown vulnerabilities across the monitored network.</td>
            </tr>
            <tr>
              <td align="left">Anti-Malware</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects and prevents the transmission of malicious
          executable code and any associated communications across the
          monitored network. This includes data exfiltration as well as
          command and control channels.</td>
            </tr>
            <tr>
              <td align="left">Anti-Spyware</td>
              <td align="left">Anti-Spyware is a subcategory of Anti Malware. Anti-Malware. Spyware transmits
          information without the user's knowledge or permission. The DUT/SUT
          detects and blocks the initial infection or transmission of data.</td>
            </tr>
            <tr>
              <td align="left">Anti-Botnet</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects and blocks traffic to or from botnets.</td>
            </tr>
            <tr>
              <td align="left">Anti-Evasion</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects and mitigates attacks that have been obfuscated
          in some manner.</td>
            </tr>
            <tr>
              <td align="left">Web Filtering</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects and blocks malicious websites websites, including defined
          classifications of websites across the monitored network.</td>
            </tr>
            <tr>
              <td align="left">DLP</td> align="left">Data Loss Protection (DLP)</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects and prevents data breaches and data exfiltration,
          or it detects and blocks the transmission of sensitive data across
          the monitored network.</td>
            </tr>
            <tr>
              <td align="left">Certificate Validation</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT validates certificates used in encrypted communications
          across the monitored network.</td>
            </tr>
            <tr>
              <td align="left">Logging and Reporting</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT logs and reports all traffic at the flow level across the
          monitored network.</td>
            </tr>
            <tr>
              <td align="left">Application Identification</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT detects known applications as defined within the traffic
          mix selected across the monitored network.</td>
            </tr>
            <tr>
              <td align="left">DPI</td> align="left">Deep Packet Inspection (DPI)</td>
              <td align="left">DUT/SUT align="left">The DUT/SUT inspects the content of the data packet.</td>
            </tr>
          </tbody>
        </table>
        <table anchor="NGFW_Security_Features" align="center">
          <name>NGFW Security Features</name>
          <thead>
            <tr>
              <th align="left">DUT/SUT (NGFW) Features</th>
              <th align="center">RECOMMENDED</th> align="center"><bcp14>RECOMMENDED</bcp14></th>
              <th align="center">OPTIONAL</th> align="center"><bcp14>OPTIONAL</bcp14></th>
            </tr>
          </thead>
          <tbody>
            <tr>
              <td align="left">TLS Inspection</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">IDS/IPS</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Spyware</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Virus</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Botnet</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Evasion</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Web Filtering</td>
              <td align="center"/>
              <td align="center">x</td>
            </tr>
            <tr>
              <td align="left">Data Loss Protection (DLP)</td>
              <td align="center"/>
              <td align="center">x</td>
            </tr>
            <tr>
              <td align="left">DDoS Protection</td>
              <td align="center"/>
              <td align="center">x</td>
            </tr>
            <tr>
              <td align="left">Certificate Validation</td>
              <td align="center"/>
              <td align="center">x</td>
            </tr>
            <tr>
              <td align="left">Application Identification</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
          </tbody>
        </table>
        <table anchor="NGIPS_Security_Features" align="center">
          <name>NGIPS Security Features</name>
          <thead>
            <tr>
              <th align="left">DUT/SUT (NGIPS) Features</th>
              <th align="center">RECOMMENDED</th> align="center"><bcp14>RECOMMENDED</bcp14></th>
              <th align="center">OPTIONAL</th> align="center"><bcp14>OPTIONAL</bcp14></th>
            </tr>
          </thead>
          <tbody>
            <tr>
              <td align="left">TLS Inspection</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Malware</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Spyware</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Botnet</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Application Identification</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Deep Packet Inspection (DPI)</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
            <tr>
              <td align="left">Anti-Evasion</td>
              <td align="center">x</td>
              <td align="center"/>
            </tr>
          </tbody>
        </table>

        <t>Note: With respect to TLS Inspection, there are scenarios where it
        will be optional.</t>
        <t>Below is a summary of the DUT/SUT configuration:</t>
        <ul spacing="normal">
          <li>
            <t>DUT/SUT MUST
          <li>The DUT/SUT <bcp14>MUST</bcp14> be configured in "inline" mode.</t>
            <t/>
          </li>
          <li>
            <t>"Fail-Open" "Inline" mode.</li>
          <li>"Fail-Open" behavior MUST <bcp14>MUST</bcp14> be disabled.</t>
            <t/>
          </li>
          <li>
            <t>All RECOMMENDED disabled.</li>
          <li>All <bcp14>RECOMMENDED</bcp14> security features are enabled.</t>
            <t/>
          </li>
          <li>
            <t>Logging enabled.</li>
          <li>Logging and reporting MUST <bcp14>MUST</bcp14> be enabled. The DUT/SUT SHOULD <bcp14>SHOULD</bcp14> log all
            traffic at the flow level (5-tuple). If the DUT/SUT is designed to
            log all traffic at different levels (e.g. (e.g., IP packet levels), it is
            acceptable to conduct tests. However, this MUST <bcp14>MUST</bcp14> be noted in the
            test report. Logging to an external device is
            permissible.</t>
            <t/>
          </li>
          <li>
            <t>Geographical
            permissible.</li>
          <li>Geographical location filtering SHOULD <bcp14>SHOULD</bcp14> be configured. If the
            DUT/SUT is not designed to perform geographical location
            filtering, it is acceptable to conduct tests without this feature.
            However, this MUST <bcp14>MUST</bcp14> be noted in the test report. </t>
            <t/>
          </li> report.</li>
          <li>Application Identification and Control MUST <bcp14>MUST</bcp14> be configured to
            trigger applications from the defined traffic mix.</li>
        </ul>
        <t>In addition, a realistic number of access control rules (ACL)
        SHOULD lists (ACLs)
        <bcp14>SHOULD</bcp14> be configured on the DUT/SUT where ACLs are configurable and
        reasonable based on the deployment scenario. For example, it is
        acceptable not to configure ACLs in an NGIPS since NGIPS devices do
        not require the use of ACLs in most deployment scenarios. This
        document determines the number of access policy rules for four
        different classes of the DUT/SUT: Extra Small (XS), Small (S), Medium (M),
        and Large (L). A sample DUT/SUT classification is described in <xref target="DUT-Classification" format="default"/>.</t>
        <t>The Access Control Rules (ACL) ACLs defined in <xref target="figure3" format="default"/>
        MUST
        <bcp14>MUST</bcp14> be configured from top to bottom in the correct order order, as shown in
        the table. This is due to ACL types listed in specificity decreasing specificity-decreasing
        order, with "block" first, followed by "allow", representing a typical
        ACL-based security policy. The ACL entries MUST <bcp14>MUST</bcp14> be configured with
        routable IP prefixes by the DUT/SUT, where applicable. (Note: There
        will be differences between how security vendors implement ACL
        decision-making.)
        decision making.) The configured ACL MUST NOT <bcp14>MUST NOT</bcp14> block the test traffic
        used for the benchmarking tests.</t>
        <figure

        <table anchor="figure3">
          <name>DUT/SUT Access List</name>
          <artwork name="" type="" align="left" alt=""><![CDATA[                                                    +---------------+
                                                    | DUT/SUT       |
                                                    | Classification|
                                                    |
	  <thead>
	    <tr>
	      <th rowspan="1" colspan="4"></th>
	      <th rowspan="1" colspan="4">DUT/SUT Classification # Rules       |
+-----------+-----------+--------------------+------+---+---+---+---+
|           | Match     |                    |      |   |   |   |   |
| Rules Type| Criteria  |   Description      |Action| XS| S | M | L |
+-------------------------------------------------------------------+
|Application|Application| Any Rules</th>
	    </tr>
	    <tr>
	      <th>Rules Type</th>
	      <th>Match Criteria</th>
	      <th>Description</th>
	      <th>Action</th>
	      <th>XS</th>
	      <th>S</th>
	      <th>M</th>
	      <th>L</th>
	    </tr>
	  </thead>
	  <tbody>
	    <tr>
	      <td>Application layer</td>
	      <td>Application</td>
	      <td>Any application    | block| 5 | 10| 20| 50|
|layer      |           | not included in    |      |   |   |   |   |
|           |           | the measurement    |      |   |   |   |   |
|           |           | traffic            |      |   |   |   |   |
+-------------------------------------------------------------------+
|Transport  |SRC traffic</td>
	      <td>block</td>
	      <td>5</td>
	      <td>10</td>
	      <td>20</td>
	      <td>50</td>
	    </tr>
	    <tr>
	      <td>Transport layer</td>
	      <td>SRC IP and | Any TCP/UDP DST ports</td>
	      <td>Any SRC IP prefix  | block| 25| 50|100|250|
|layer      |TCP/UDP    | used and any DST   |      |   |   |   |   |
|           |DST ports  | ports not used in  |      |   |   |   |   |
|           |           | the measurement    |      |   |   |   |   |
|           |           | traffic            |      |   |   |   |   |
+-------------------------------------------------------------------+
|IP layer   |SRC/DST IP | Any traffic</td>
	      <td>block</td>
	      <td>25</td>
	      <td>50</td>
	      <td>100</td>
	      <td>250</td>
	    </tr>
	    <tr>
	      <td>IP layer</td>
	      <td>SRC/DST IP</td>
	      <td>Any SRC/DST IP     | block| 25| 50|100|250|
|           |           | subnet not used    |      |   |   |   |   |
|           |           | in the measurement |      |   |   |   |   |
|           |           | traffic            |      |   |   |   |   |
+-------------------------------------------------------------------+
|Application|Application| Half traffic</td>
	      <td>block</td>
	      <td>25</td>
	      <td>50</td>
	      <td>100</td>
	      <td>250</td>
	    </tr>
	    <tr>
	      <td>Application layer</td>
	      <td>Application</td>
	      <td>Half of the        | allow| 10| 10| 10| 10|
|layer      |           | applications       |      |   |   |   |   |
|           |           | included in the    |      |   |   |   |   |
|           |           | measurement traffic|      |   |   |   |   |
|           |           |(see traffic (see the note below)|      |   |   |   |   |
+-------------------------------------------------------------------+
|Transport  |SRC below)</td>
	      <td>allow</td>
	      <td>10</td>
	      <td>10</td>
	      <td>10</td>
	      <td>10</td>
	    </tr>
	    <tr>
	      <td>Transport layer</td>
	      <td>SRC IP and | Half TCP/UDP DST ports</td>
	      <td>Half of the SRC    | allow| >1| >1| >1| >1|
|layer      |TCP/UDP    | IPs used and any   |      |   |   |   |   |
|           |DST ports  | DST ports used in  |      |   |   |   |   |
|           |           | the measurement    |      |   |   |   |   |
|           |           | traffic            |      |   |   |   |   |
|           |           | (one rule per      |      |   |   |   |   |
|           |           | subnet)            |      |   |   |   |   |
+-------------------------------------------------------------------+
|IP layer   |SRC IP     | The subnet)</td>
	      <td>allow</td>
	      <td>>1</td>
	      <td>>1</td>
	      <td>>1</td>
	      <td>>1</td>
	    </tr>
	    <tr>
	      <td>IP layer</td>
	      <td>SRC IP</td>
	      <td>The rest of the    | allow| >1| >1| >1| >1|
|           |           | SRC IP prefix      |      |   |   |   |   |
|           |           | range used in the  |      |   |   |   |   |
|           |           | measurement        |      |   |   |   |   |
|           |           | measurment traffic            |      |   |   |   |   |
|           |           | (one rule per      |      |   |   |   |   |
|           |           | subnet)            |      |   |   |   |   |
+-----------+-----------+--------------------+------+---+---+---+---+]]></artwork>
        </figure> subnet)</td>
	      <td>allow</td>
	      <td>>1</td>
	      <td>>1</td>
	      <td>>1</td>
	      <td>>1</td>
	    </tr>
	  </tbody>
	</table>
        <t>Note 1: Based on the test customer's specific use case, the testers
        can increase the number of rules.</t>
        <t>Note 2: If half of the applications included in the test traffic is are
        less than 10, the missing number of ACL entries (dummy (placeholder rules) can be
        configured for any application traffic not included in the test
        traffic.</t>
        <t>Note 3: In the event, event that the DUT/SUT is designed to not use ACLs ACLs, it is
        acceptable to conduct tests without them. However, this MUST <bcp14>MUST</bcp14> be noted
        in the test report.</t>
        <section anchor="security_effectiveness" numbered="true" toc="default">
          <name>Security Effectiveness Configuration</name>
          <t>The selected security features (defined in Tables <xref target="NGFW_Security_Features" format="default"/> format="counter"/> and <xref target="NGIPS_Security_Features" format="default"/>) format="counter"/>) of the DUT/SUT MUST <bcp14>MUST</bcp14> be
          configured effectively to detect, prevent, and report the defined
          security vulnerability sets. This section defines the selection of
          the security vulnerability sets from the Common Vulnerabilities and
          Exposures (CVE) (CVEs) list <xref target="CVE" format="default"/> for testing. The vulnerability set should
          reflect a minimum of 500 CVEs from no older than 10 calendar years
          to the current year. These CVEs should be selected with a focus on
          in-use software commonly found in business applications, with a
          Common Vulnerability Scoring System (CVSS) Severity of High
          (7-10).</t>
          <t>This document is primarily focused on performance benchmarking.
          However, it is RECOMMENDED <bcp14>RECOMMENDED</bcp14> to validate the security features
          configuration of the DUT/SUT by evaluating the security
          effectiveness as a prerequisite for performance benchmarking tests
          defined in section 7. <xref target="Benchmarking" format="default"/>. In case the benchmarking tests are performed
          without evaluating security effectiveness, the test report MUST <bcp14>MUST</bcp14>
          explain the implications of this. The methodology for evaluating
          security effectiveness is defined in <xref target="Test-Methodology-Security-Effectiveness-Evaluation" format="default"/>.</t>
        </section>
      </section>
      <section anchor="Test_Equipment_Configuration" numbered="true" toc="default">
        <name>Test Equipment Configuration</name>
        <t>In general, test equipment allows configuring parameters in
        different protocol layers. Extensive proof of concept proof-of-concept tests conducted
        to support preparation of this document showed that benchmarking
        results are strongly affected by the choice of protocol stack
        parameters;
        parameters, especially OSI layer 4 transport protocol parameters. For
        more information on how TCP and QUIC parameters will impact
        performance
        performance, review <xref target="fastly" format="default"/>. To achieve reproducible
        results that will be representative for of real deployment scenarios,
        careful specification and documentation of the parameters are
        required.</t>
        <t>This section specifies common test equipment configuration
        parameters applicable for all benchmarking tests defined in <xref target="Benchmarking" format="default"/>. Any benchmarking test specific benchmarking-test-specific
        parameters are described under the test setup section of each
        benchmarking test individually.</t>
        <section numbered="true" toc="default">
          <name>Client Configuration</name>
          <t>This section specifies which parameters should be considered
          while configuring emulated client endpoints in the test equipment.
          Also, this section specifies the RECOMMENDED <bcp14>RECOMMENDED</bcp14> values for certain
          parameters. The values are the defaults typically used in most of
          the client operating system types.</t>
          <t>Pre-standard evaluations have shown that it is possible to set a
          wide range of arbitrary parameters for OSI layer 4 transport
          protocols on test equipment leading to optimization of client-specific results
          optimization; results;
          however, only well-defined common parameter sets help
          to establish meaningful and comparable benchmarking results. For
          these reasons, this document recommends specific sets of transport
          protocol parameters to be configured on test equipment used for
          benchmarking.</t>
          <section anchor="TCP_Stack_client" numbered="true" toc="default">
            <name>TCP Stack Attributes</name>
            <t>The TCP stack of the emulated client endpoints MUST
            <bcp14>MUST</bcp14> fulfill the TCP requirements defined in <xref
            target="RFC9293" format="default"/> (See Appendix
            B.). sectionFormat="of" section="B"/>. In addition,
            this section specifies the RECOMMENDED <bcp14>RECOMMENDED</bcp14> values for
            TCP parameters configured using the following parameters:</t> parameters described below.</t>
            <t>The IPv4 and IPv6 Maximum Segment Size (MSS) Sizes (MSSs) are set to 1460
            bytes and 1440 bytes bytes, respectively. TX and RX initial receive
            window sizes are set to 65535 bytes. The client's initial
            congestion window should not exceed 10 times the MSS. Delayed ACKs
            are permitted permitted, and the maximum client delayed ACK should not exceed
            10 times of the MSS before a forced ACK ACK; also, the maximum delayed
            ACK timer is allowed to be set to 200 ms. Up to three retries are
            allowed before a timeout event is declared. The TCP PSH flag is set to
            high in all traffic. The source port range is in the range of 1024
            - 65535.  1024-65535.
	    The clients initiate TCP connections via a three-way
            handshake (SYN, SYN/ACK, ACK) and close TCP connections via either
            a TCP three-way close (FIN, FIN/ACK, ACK) or a TCP four-way close
            (FIN, ACK, FIN, ACK).</t>
          </section>
          <section anchor="QUIC_Spec_Client" numbered="true" toc="default">
            <name>QUIC Specification</name>
            <t>QUIC stack emulation on the test equipment MUST <bcp14>MUST</bcp14>
            conform to <xref target="RFC9000" format="default"/> and <xref
            target="RFC9001" format="default"/>. This section specifies the RECOMMENDED
            <bcp14>RECOMMENDED</bcp14> values for certain QUIC parameters to
            be configured on test equipment used for benchmarking purposes
            only. The QUIC Stream stream type (defined in section
            2.1 of <xref target="RFC9000" format="default"/>)
            sectionFormat="of" section="2.1"/>) is set to "Client-Initiated,
            Bidirectional". 0-RTT and early data are Disabled. disabled. The QUIC Connection connection
            termination method is Immediate an immediate close (section 10.2 of <xref (<xref target="RFC9000" format="default"/>.
            sectionFormat="of" section="10.2"/>). Flow control is enabled. UDP
            payloads are set to the datagram size of 1232 bytes for IPv6 and 1252
            bytes for IPv4.  In addition, transport parameters and default
            values defined in
            section 18.2 of <xref target="RFC9000" format="default"/> sectionFormat="of"
            section="18.2"/> are RECOMMENDED <bcp14>RECOMMENDED</bcp14> to configure on
            test equipment. Also, this document references
            Appendixes B.1 Appendices <xref target="RFC9002"
	    section="B.1" sectionFormat="bare"/> and B.2
            <xref target="RFC9002" section="B.2" sectionFormat="bare"/> of <xref
	    target="RFC9002" format="default"/> for congestion
            control related congestion-control-related
	    constants and variables. Any configured QUIC and
            UDP parameter(s) MUST parameter <bcp14>MUST</bcp14> be documented in the test
            report.</t>
          </section>
          <section anchor="Client_IP" numbered="true" toc="default">
            <name>Client IP Address Space</name>
            <t>The client IP space contains the following attributes.</t>
            <ul spacing="normal">
              <li>If multiple IP blocks are used, they MUST be <bcp14>MUST</bcp14> consist of
                multiple unique, discontinuous static address blocks.</li>
		<li>A default gateway MAY <bcp14>MAY</bcp14> be used.</li>
              <li>The DSCP (differentiated differentiated services code point) point (DSCP) marking
                should be set to DF (Default Forwarding) Default Forwarding (DF) '000000' on the IPv4 Type
                of Service (ToS) field and IPv6 traffic class Traffic Class field.</li>
              <li>Extension header(s) MAY
              <li>One or more extension headers <bcp14>MAY</bcp14> be used for IPv6 clients. If
                multiple extension headers are needed for traffic emulation,
                this document references <xref target="RFC8200" format="default"/> to choose
                the correct order of the extension headers within an IPv6
                packet. Testing with one or more extension header(s) headers may impact the
                performance of the DUT. The extension headers MUST <bcp14>MUST</bcp14> be
                documented and reported.</li>
            </ul>
            <t>The following equation can be used to define the total number
            of client IP addresses that need to be configured on the test
            equipment.</t>
            <t>Desired
	    <t indent="3">Desired total number of client IP addresses = Target throughput
            [Mbit/s] / Average throughput per IP address [Mbit/s]</t>
            <t>As shown in the example list below, the value for "Average
            throughput per IP address" can be varied depending on the
            deployment and use case scenario.</t>
            <ol spacing="normal" type="(Example %d)"><li>DUT/SUT type="Example %d">
	      <li>DUT/SUT deployment scenario 1 : 1: 6-7 Mbit/s per IP (e.g. (e.g.,
                1,400-1,700 IPs per 10Gbit/s 10 Gbit/s of throughput)</li>
              <li>DUT/SUT deployment scenario 2 : 2: 0.1-0.2 Mbit/s per IP (e.g. (e.g.,
                50,000-100,000 IPs per 10Gbit/s 10 Gbit/s of throughput)</li>
            </ol>
            <t>Client IP addresses MUST <bcp14>MUST</bcp14> be distributed between IPv4 and IPv6
            based on the deployment and use case scenario. The following options
            MAY
            <bcp14>MAY</bcp14> be considered for a selection of ratios for both IP addresses
            and traffic load distribution.</t>
            <ol spacing="normal" type="(Option %d)"><li>100 type="Option %d">
	      <li>100 % IPv4, no IPv6</li>
              <li>80 % IPv4, 20% IPv6</li>
              <li>50 % IPv4, 50% IPv6</li>
              <li>20 % IPv4, 80% IPv6</li>
              <li>no IPv4, 100% IPv6</li>
            </ol>
            <t>Note: IANA has assigned IP address ranges for testing purposes purposes,
            as described in <xref target="IANA" format="default"/>. If the test scenario
            requires more IP addresses or subnets than IANA has assigned, this
            document recommends using private IPv4 address ranges or Unique
            Local Address (ULA) IPv6 address ranges for the testing.</t>
          </section>
          <section anchor="Emulated_web_Browser_attributes" numbered="true" toc="default">
            <name>Emulated Web Browser Attributes</name>
            <t>The client (emulated web browser) contains attributes that will
            materially affect the traffic load. The objective is to emulate
            modern, typical browser attributes to improve the relevance of the
            result set for typical deployment scenarios.</t>
            <t>The emulated browser MUST <bcp14>MUST</bcp14> negotiate HTTP version 1.1 or higher.
            The emulated browser SHOULD <bcp14>SHOULD</bcp14> advertise a User-Agent header. The
            emulated browser MUST <bcp14>MUST</bcp14> enforce content length validation. HTTP
            header compression MAY <bcp14>MAY</bcp14> be set to enable. If HTTP header
            compression is configurable in the test equipment, it MUST <bcp14>MUST</bcp14> be
            documented if it was enabled or disabled. Depending on test
            scenarios and the chosen HTTP version, the emulated browser MAY <bcp14>MAY</bcp14> open
            multiple TCP or QUIC connections per Server server endpoint IP at any
            time
            time, depending on how many sequential transactions need to be
            processed.</t>
            <t>For HTTP/2 traffic emulation, the emulated browser opens
            multiple concurrent streams per connection (multiplexing). For
            HTTPS requests, the emulated browser MUST <bcp14>MUST</bcp14> send an "h2" protocol
            identifier using the TLS extension Application Layer Application-Layer Protocol
            Negotiation (ALPN). The following default values (see <xref target="Undertow" format="default"/>) are the RECOMMENDED setting <bcp14>RECOMMENDED</bcp14> settings for certain
            HTTP/2 parameters to be configured on test equipment used for
            benchmarking purposes only:</t>
            <ul spacing="normal">
              <li>Maximum Frame frame size: 16384 bytes</li>
              <li>Initial Window window size: 65535 bytes</li>
              <li>HPACK Header header field table size: 4096 bytes</li>
              <li>Server PUSH push enable: false (Note: in In <xref target="Undertow" format="default"/> format="default"/>, the default setting is true. However, for
                testing purposes, this document recommends setting the value to
                false for server push.)</li>
            </ul>
            <t>This document refers to <xref target="RFC9113" format="default"/> for further
            details of HTTP/2. If any additional parameters are used to
            configure the test equipment, they MUST <bcp14>MUST</bcp14> be documented.</t>
            <t>For HTTP/3 traffic emulation, the emulated browsers initiate
            secure QUIC connections using TLS 1.3 (<xref target="RFC9001" format="default"/>
            describes how TLS is used to secure QUIC). This document refers to
            <xref target="RFC9114" format="default"/> for HTTP/3 specifications. The
            specification for transport protocol parameters is defined in
            <xref target="QUIC_Spec_Client" format="default"/>. QPACK
            configuration settings settings, such as MAX_TABLE_CAPACITY and
            QPACK_BLOCKED_STREAMS
            QPACK_BLOCKED_STREAMS, are set to zero (default) (default), as defined in
            <xref target="RFC9204" format="default"/>. Any HTTP/3 parameters used for test
            equipment configuration MUST <bcp14>MUST</bcp14> be documented.</t>
            <t>For encrypted traffic, the following attributes are defined as
            the negotiated encryption parameters. The test clients MUST <bcp14>MUST</bcp14> use
            TLS version 1.2 or higher. The TLS record size MAY <bcp14>MAY</bcp14> be optimized
            for the HTTPS response object size size, up to a record size of 16
            KBytes.
            KB. If Server Name Indication (SNI) is required (especially if
            the server is identified by a domain name), the client endpoint
            MUST
            <bcp14>MUST</bcp14> send TLS extension Server Name Indication (SNI) SNI information
            when opening a security tunnel. Each client connection MUST <bcp14>MUST</bcp14>
            perform a full TLS handshake handshake, and session reuse or resumption MUST <bcp14>MUST</bcp14>
            be disabled. (Note: Real web browsers use session reuse or
            resumption. However, for testing purposes, this feature must not
            be used to measure the DUT/SUT performance in the worst-case
            scenario.)</t>
            <t>The following TLS 1.2 supported ciphers and keys supported by TLS 1.2 are
            RECOMMENDED
            <bcp14>RECOMMENDED</bcp14> for HTTPS based the HTTPS-based benchmarking tests defined in <xref target="Benchmarking" format="default"/>.</t>
            <ol spacing="normal" type="1"><li>ECDHE-ECDSA-AES128-GCM-SHA256 type="1">
	      <li>ECDHE-ECDSA-AES128-GCM-SHA256 with Prime256v1 (Signature
                Hash Algorithm: ecdsa_secp256r1_sha256 and Supported group:
                secp256r1)</li>
              <li>ECDHE-RSA-AES128-GCM-SHA256 with RSA 2048 (Signature Hash
                Algorithm: rsa_pkcs1_sha256 and Supported group:
                secp256r1)</li>
              <li>ECDHE-ECDSA-AES256-GCM-SHA384 with Secp384r1 (Signature
                Hash Algorithm: ecdsa_secp384r1_sha384 and Supported group:
                secp384r1)</li>
              <li>ECDHE-RSA-AES256-GCM-SHA384 with RSA 4096 (Signature Hash
                Algorithm: rsa_pkcs1_sha384 and Supported group:
                secp384r1)</li>
            </ol>
            <t>Note: The above ciphers and keys were those commonly used for
            enterprise-grade encryption cipher suites for TLS 1.2 as at of the
            time of publication (2022). (2023). Individual certification bodies should
            use ciphers and keys that reflect evolving use cases. These
            choices MUST <bcp14>MUST</bcp14> be documented in the resulting test reports with
            detailed information on the ciphers and keys used used, along with
            reasons for the choices.</t>
            <t>IANA recommends the following cipher suites for use with TLS
            1.3
            1.3, as defined in <xref target="RFC8446" format="default"/>.</t>
            <ol spacing="normal" type="1"><li>TLS_AES_128_GCM_SHA256</li> type="1">
	      <li>TLS_AES_128_GCM_SHA256</li>
              <li>TLS_AES_256_GCM_SHA384</li>
              <li>TLS_CHACHA20_POLY1305_SHA256</li>
              <li>TLS_AES_128_CCM_SHA256</li>
            </ol>
          </section>
        </section>
        <section numbered="true" toc="default">
          <name>Backend Server Configuration</name>
          <t>This section specifies which parameters should be considered
          while configuring emulated backend servers using test equipment.</t>
          <section numbered="true" toc="default">
            <name>TCP Stack Attributes</name>
            <t>The TCP stack on the server-side MUST <bcp14>MUST</bcp14> be configured similar similarly to
            the client-side configuration described in <xref target="TCP_Stack_client" format="default"/></t> format="default"/>.</t>
          </section>
          <section anchor="QUIC_Spec_Server" numbered="true" toc="default">
            <name>QUIC Specification</name>
            <t>The QUIC parameters on the server-side MUST <bcp14>MUST</bcp14> be configured
            similar
            similarly to the client-side configuration. Any configured QUIC
            Parameter(s) MUST
            parameter <bcp14>MUST</bcp14> be documented in the report.</t>
          </section>
          <section anchor="Server_IP" numbered="true" toc="default">
            <name>Server Endpoint IP Addressing</name>
            <t>The sum of the server IP space MUST <bcp14>MUST</bcp14> contain the following
            attributes.</t>
            <ul spacing="normal">
              <li>The server IP blocks MUST <bcp14>MUST</bcp14> consist of unique, discontinuous
                static address blocks with one IP per server Fully Qualified
                Domain Name (FQDN) endpoint per test port.</li>
              <li>A default gateway is permitted. The
                DSCP (differentiated
                services code point) marking is set to DF (Default Forwarding)
                '000000' on the IPv4 Type of Service (ToS) ToS field and IPv6 traffic
                class Traffic
                Class field. Extension header(s) One or more extension headers for the IPv6 server is are
                permitted. If multiple extension headers are required, this
                document referenced references <xref target="RFC8200" format="default"/> to choose the
                correct order of the extension headers within an IPv6
                packet.</li>
              <li>The server IP address distribution between IPv4 and IPv6
                MUST
                <bcp14>MUST</bcp14> be identical to the client IP address distribution
                ratio.</li>
            </ul>
            <t>Note: The IANA has assigned IP address blocks for the testing
            purpose as described in <xref target="IANA" format="default"/>. If the test
            scenario requires more IP addresses or address blocks than the
            IANA has assigned, this document recommends using private IPv4 address
            ranges or Unique Local Address (ULA) IPv6 address ranges for the
            testing.</t>
          </section>
          <section numbered="true" toc="default">
            <name>HTTP / HTTPS
            <name>HTTP/HTTPS Server Pool Endpoint Attributes</name>
            <t>The HTTP 1.1 and HTTP/2 server pools listen on TCP ports 80 and
            443 for HTTP and HTTPS.
	    The HTTP/3 server pool listens on UDP port 443
            or any UDP port. The server MUST <bcp14>MUST</bcp14> emulate the same HTTP version (HTTP
            1.1 or HTTP/2
            1.1, HTTP/2, or HTTP/3) and settings chosen by the client
            (emulated web browser). For the HTTPS server, TLS version 1.2 or higher
            MUST
            <bcp14>MUST</bcp14> be used with a maximum record size of 16 KByte. KB. Ticket
            resumption or session ID reuse MUST NOT <bcp14>MUST NOT</bcp14> be used for TLS 1.2 and
            also 1.2;
            also, session Ticket ticket or session cache MUST NOT <bcp14>MUST NOT</bcp14> be used for TLS 1.3.
            The server MUST <bcp14>MUST</bcp14> serve a certificate to the client. Cipher The cipher suite
            and key size on the server-side MUST <bcp14>MUST</bcp14> be configured similar similarly to the
            client-side configuration described in <xref target="Emulated_web_Browser_attributes" format="default"/>.</t>
          </section>
        </section>
        <section numbered="true" toc="default">
          <name>Traffic Flow Definition</name>
          <t>This section describes the traffic pattern between client and
          server endpoints.
          <t>   At the beginning of the test, test (the init phase; see <xref target="Traffic_Load_Profile"/>),
   the server endpoint
          initializes initializes, and the server endpoint will be
   ready to accept connection states including
          initialization of the TCP or QUIC stack connections as well as bound inbound HTTP and
   HTTPS servers. When a requests.  The client endpoint is needed, it will endpoints initialize and be are given
   attributes such as a MAC and IP address. The behavior After the init phase of
   the test, each client is to sweep sweeps through the given server IP space,
   generating a recognizable service recognizable by the DUT.  Sequential and
          pseudorandom sweep methods are acceptable. The method used MUST <bcp14>MUST</bcp14> be
          stated in the final report. Thus, a balanced mesh between client
          endpoints and server endpoints will be generated in a client IP and
          port to server IP and port combination. Each client endpoint
          performs the same actions as other endpoints, with the difference
          being the source IP of the client endpoint and the target server IP
          pool. The client MUST <bcp14>MUST</bcp14> use the server IP address or FQDN in the host
          header.</t>
          <section numbered="true" toc="default">
            <name>Description of Intra-Client Behavior</name>
            <t>Client endpoints are independent of other clients that are
            concurrently executing. When a client endpoint initiates traffic,
            this section describes how the client steps through different
            services. Once the test is initialized, the client endpoints
            randomly hold (perform no operation) for a few milliseconds for
            better randomization of the start of client traffic. Each client
            (HTTP 1.1 or HTTP/2) will either open a new TCP connection or
            connect to an HTTP persistent connection that is still open to that
            specific server. HTTP/3 clients will open UDP streams within QUIC
            connections. At any point that the traffic profile may require
            encryption, a TLS encryption tunnel will form form, presenting the URL
            or IP address request to the server.    If using SNI, the server MUST
	    <bcp14>MUST</bcp14> then perform an SNI name check with
	    by comparing the proposed FQDN compared to the domain embedded in
	    the certificate. Only when correct, correct will
            the server process the HTTPS response object. The initial response
            object to the server is based on benchmarking tests described in
            <xref target="Benchmarking" format="default"/>. Multiple additional
            sub-URLs (response objects on the service page) MAY <bcp14>MAY</bcp14> be requested
            simultaneously. This MAY <bcp14>MAY</bcp14> be to the same server IP as the initial
            URL. Each sub-object will also use a canonical FQDN and URL
            path.</t>
          </section>
        </section>
        <section anchor="Traffic_Load_Profile" numbered="true" toc="default">
          <name>Traffic Load Profile</name>
          <t>The loading of traffic is described in this section. The loading
          of a traffic load profile has five phases: Init, ramp up, sustain,
          ramp down, and collection.</t>
          <ol spacing="normal" type="1"><li>Init phase: Testbed devices
          <dl newline="true" spacing="normal">
	    <dt>Init phase:</dt>
	    <dd>Testbed devices, including the client and server
              endpoints
              endpoints, should negotiate layer 2-3 connectivity connectivity, such as MAC
              learning and ARP/ND. Only after successful MAC learning or
              ARP/ND SHALL <bcp14>SHALL</bcp14> the test iteration move to the next phase. No
              measurements are made in this phase. The minimum recommended
              time for the Init phase is 5 seconds. During this phase, the
              emulated clients MUST NOT <bcp14>MUST NOT</bcp14> initiate any sessions with the
              DUT/SUT,
              DUT/SUT; in contrast, the emulated servers should be ready to
              accept requests from the DUT/SUT or emulated clients.</li>
            <li>Ramp up phase: The clients.</dd>
            <dt>Ramp Up phase:</dt>
	    <dd>The test equipment MUST <bcp14>MUST</bcp14> start to generate the
	    test traffic. It MUST <bcp14>MUST</bcp14> use a set of the approximate
	    number of unique client IP addresses to generate traffic. The
	    traffic MUST <bcp14>MUST</bcp14> ramp up from zero to the desired target
	    objective. The target objective is defined for each benchmarking
	    test. The duration for the ramp up phase MUST <bcp14>MUST</bcp14> be
	    configured long enough that the test equipment does not overwhelm
	    the DUT/SUTs DUT's/SUT's stated performance metrics defined in <xref
	    target="Key_Performance_Indicators" format="default"/> namely, format="default"/>, namely TCP or QUIC
              Connections Per Second, Inspected Throughput, Concurrent
	    connections per second, inspected throughput, concurrent
	    TCP or QUIC Connections, connections, and Application Transactions Per Second. application transactions per
	    second. No measurements are made in this phase.</li>
            <li>Sustain phase: Starts phase.</dd>
            <dt>Sustain phase:</dt>
	    <dd>This phase starts when all required clients are active
              and operating at their desired load condition. In the sustain
              phase, the test equipment MUST <bcp14>MUST</bcp14> continue generating traffic to a
              constant target value for a constant number of active clients.
              The minimum RECOMMENDED <bcp14>RECOMMENDED</bcp14> time duration for the sustain phase is 300
              seconds. This is the phase where measurements occur. The test
              equipment MUST <bcp14>MUST</bcp14> measure and record statistics continuously. The
              sampling interval for collecting the raw results and calculating
              the statistics MUST <bcp14>MUST</bcp14> be less than 2 seconds.</li>
            <li>Ramp down phase: The seconds.</dd>
            <dt>Ramp Down phase:</dt>
	    <dd>The test traffic slows down from the target
              number to 0, and no measurements are made.</li>
            <li>Collection phase: The made.</dd>
            <dt>Collection phase:</dt>
	    <dd>The last phase is administrative and will
              occur when the test equipment merges and collates the report
              data.</li>
          </ol>
              data.</dd>
          </dl>
        </section>
      </section>
    </section>
    <section numbered="true" toc="default">
      <name>Testbed Considerations</name>
      <t>This section describes steps for a reference test (pre-test) that
      control
      controls the test environment environment, including test equipment, focusing on
      physical and virtualized environments and as well as test equipment.
      Below are the RECOMMENDED <bcp14>RECOMMENDED</bcp14> steps for the reference test.</t>
      <ol spacing="normal" type="1"><li>Perform type="1">
	<li>Perform the reference test either by configuring the DUT/SUT in
          the most trivial setup (fast forwarding) or without the presence of
          the DUT/SUT.</li>
        <li>Generate traffic from the traffic generator. Choose a traffic profile
          used for the HTTP or HTTPS throughput performance test with the
          smallest object size.</li>
          <li>Ensure that any ancillary switching or routing functions added in
       the test equipment do not limit the performance by introducing
          network metrics such as
       packet loss and or latency. This is
          specifically important for virtualized components (e.g., vSwitches, vSwitches or
          vRouters).</li>
        <li>Verify that the generated traffic (performance) of the test
          equipment matches and reasonably exceeds the expected maximum
          performance of the DUT/SUT.</li>
        <li>Record the network performance metrics packet loss and latency
          introduced by the test environment (without the DUT/SUT).</li>
        <li>Assert that the testbed characteristics are stable during the
          entire test session. Several factors might influence stability
          specifically, stability,
          specifically for virtualized testbeds. For testbeds, for example, additional
          workloads in a virtualized system, load balancing, and movement of
          virtual machines during the test, test or simple issues issues, such as
          additional heat created by high workloads leading to an emergency
          CPU performance reduction.</li>
      </ol>
      <t>The reference test MUST <bcp14>MUST</bcp14> be performed before the benchmarking tests
      (described in section 7) <xref target="Benchmarking" format="default"/>) start.</t>
    </section>
    <section numbered="true" toc="default">
      <name>Reporting</name>
      <t>This section describes how the benchmarking test report should be
      formatted and presented. It is RECOMMENDED <bcp14>RECOMMENDED</bcp14> to include two main sections
      in the report: the introduction and the detailed test results
      sections.</t>
      <section numbered="true" toc="default">
        <name>Introduction</name>
        <t>The following attributes should be present in the introduction
        section of the test report.</t>
        <ol spacing="normal" type="1"><li>The time type="1">
	  <li>Time and date of the execution of the tests</li>
          <li>
            <t>Summary of testbed software and hardware details</t>
            <ol spacing="normal" type="a"><li>
                <t>DUT/SUT hardware/virtual configuration</t>
                <ul spacing="normal">
                  <li>This section should clearly identify the make
                  <li>Make and model
                    of the DUT/SUT</li>
                  <li>The port DUT/SUT, which should be clearly identified</li>
                  <li>Port interfaces, including speed and link
                  information</li>
                  <li>If the DUT/SUT is a Virtual Network Function (VNF),
                    host (VNF)</li>
                  <li>Host (server) hardware and software details, interface details</li>
		  <li>Interface acceleration type such (such as DPDK and SR-IOV, used Data Plane Development Kit (DPDK) and single-root input/output
		  virtualization (SR-IOV))</li>
		  <li>Used CPU cores,
                    used RAM, resource cores</li>
                  <li>Used RAM</li>
		  <li>Resource sharing (e.g. Pinning (e.g., pinning details and NUMA
                    Node) Non-Uniform Memory Access (NUMA)
                  node) configuration details, hypervisor version, virtual details</li>
		  <li>Hypervisor version</li>
		  <li>Virtual switch version</li>
                  <li>details
                  <li>Details of any additional hardware relevant to the
                    DUT/SUT
                    DUT/SUT, such as controllers</li>
                </ul>
              </li>
              <li>
                <t>DUT/SUT software</t>
                <ul spacing="normal">
                  <li>Operating system name</li>
                  <li>Version</li>
                  <li>Specific configuration details (if any)</li>
                </ul>
              </li>
              <li>
                <t>DUT/SUT enabled
                <t>DUT-/SUT-enabled features</t>
                <ul spacing="normal">
                  <li>Configured DUT/SUT features (see Tables <xref target="NGFW_Security_Features" format="default"/> format="counter"/> and <xref target="NGIPS_Security_Features" format="default"/>)</li> format="counter"/>)</li>
                  <li>Attributes of the above-mentioned abovementioned features</li>
                  <li>Any additional relevant information about the
                    features</li>
                </ul>
              </li>
              <li>
                <t>Test equipment hardware and software </t>
                <ul spacing="normal">
                  <li>Test equipment vendor name</li>
                  <li>Hardware details details, including model number, number and interface
                    type</li>
                  <li>Test equipment firmware and test application software
                    version</li>
                    <li>If the test equipment is a virtual solution, the solution</li>
		    <li>The host
                    (server) hardware and software details, interface details</li>
		    <li>Interface acceleration type such (such as DPDK and SR-IOV, used SR-IOV)</li>
		    <li>Used CPU cores,
                    used RAM, resource cores</li>
                    <li>Used RAM</li>
		    <li>Resource sharing (e.g. Pinning (e.g., pinning details and NUMA
                    Node)
                    node) configuration details, hypervisor version, virtual details</li>
		    <li>Hypervisor version</li>
		    <li>Virtual switch version</li>
                </ul>
              </li>
              <li>
                <t>Key test parameters</t>
                <ul spacing="normal">
                  <li>Used cipher suites and keys</li>
                  <li>IPv4 and IPv6 traffic distribution</li>
                  <li>Number of configured ACL</li>
                  <li>TCP, ACLs</li>
                  <li>TCP and UDP stack parameter parameter, if tested</li>
                  <li>QUIC, HTTP/2, and HTTP/3 parameters parameters, if tested</li>
                </ul>
              </li>
              <li>
                <t>Details of the application traffic mix used in the benchmarking
                test <xref format="default" target="Throughput_Performance_With_Traffic_Mix">"Throughput target="Throughput_Performance_With_Traffic_Mix">Throughput
                Performance with Application Traffic Mix"</xref></t> Mix</xref></t>
                <ul spacing="normal">
                  <li>Name of applications and layer 7 protocols</li>
                  <li>Percentage of emulated traffic for each application and
                    layer 7 protocols</li>
                  <li>Percentage of encrypted traffic and traffic, used cipher suites suites,
                    and keys (The RECOMMENDED (the <bcp14>RECOMMENDED</bcp14> ciphers and keys are defined in
                    <xref target="Emulated_web_Browser_attributes" format="default"/>)</li>
                  <li>Used object sizes for each application and layer 7
                    protocols</li>
                </ul>
              </li>
            </ol>
          </li>
          <li>
            <t>Results Summary / Executive Summary</t>
            <ol spacing="normal" type="a"><li>Results type="a">
	      <li>Results should be presented with an introduction section
                documenting the summary of results in a prominent, easy to
                read easy-to-read block.</li>
            </ol>
          </li>
        </ol>
      </section>
      <section numbered="true" toc="default">
        <name>Detailed Test Results</name>
        <t>In the result results section of the test report, the following attributes
        should be present for each benchmarking test.</t>
        <ol spacing="normal" type="a"><li>KPIs MUST type="a">
	  <li>KPIs <bcp14>MUST</bcp14> be documented separately for each benchmarking test.
            The format of the KPI metrics MUST <bcp14>MUST</bcp14> be presented as described in
            <xref target="Key_Performance_Indicators" format="default"/>.</li>
          <li>The next level of detail details should be graphs showing each of these
            metrics over the duration (sustain phase) of the test. This allows
            the user to see the measured performance stability changes over
            time.</li>
        </ol>
      </section>
      <section anchor="Key_Performance_Indicators" numbered="true" toc="default">
        <name>Benchmarks and Key Performance Indicators</name>
        <t>This section lists key performance indicators (KPIs) for overall
        benchmarking tests. All KPIs MUST <bcp14>MUST</bcp14> be measured during the sustain phase
        of the traffic load profile described in <xref target="Traffic_Load_Profile" format="default"/>. Also, the KPIs MUST <bcp14>MUST</bcp14> be measured from
        the result output of test equipment.</t>
        <ul
        <dl newline="true" spacing="normal">
          <li>
            <t>Concurrent
          <dt>Concurrent TCP Connections</t>
            <t>The Connections</dt>
            <dd>The aggregate number of simultaneous connections between hosts
            across the DUT/SUT, DUT/SUT or between hosts and the DUT/SUT (defined in
            <xref target="RFC2647" format="default"/>).</t>
          </li>
          <li>
            <t>Concurrent format="default"/>).</dd>
          <dt>Concurrent QUIC Connections</t>
            <t>The Connections</dt>
            <dd>The aggregate number of simultaneous connections between hosts
            across the DUT/SUT.</t>
          </li>
          <li>
            <t>TCP DUT/SUT.</dd>
          <dt>TCP Connections Per Second</t>
            <t>The Second</dt>
            <dd>The average number of successfully established TCP connections
            per second between hosts across the DUT/SUT, DUT/SUT or between hosts and
            the DUT/SUT. As described in <xref target="TCP_Stack_client"
            format="default"/>, the TCP connections are initiated by clients
            via a TCP three-way handshake (SYN, SYN/ACK, ACK). Then Then, the TCP
            session data is sent sent, and then the TCP sessions are closed via
            either a TCP three-way close (FIN, FIN/ACK, ACK) or a TCP four-way
            close (FIN, ACK, FIN, ACK). The TCP sessions MUST
            NOT <bcp14>MUST
            NOT</bcp14> be closed by RST.</t>
          </li>
          <li>
            <t>QUIC RST.</dd>
          <dt>QUIC Connections Per Second</t>
            <t>The Second</dt>
            <dd><t>The average number of successfully established QUIC
            connections per second between hosts across the DUT/SUT. As
            described in <xref target="QUIC_Spec_Client" format="default"/>,
            the QUIC connections are initiated by clients. Then Then, the data is sent
            sent, and then the QUIC sessions are closed by the "immediate close"
            method.</t>
            <t>Since the QUIC specification defined in <xref
            target="QUIC_Spec_Client" format="default"/> recommends disabling
            0-RTT and early data, this KPI is focused on the
 1-RTT handshake. If
            required, 0-RTT can be also measured in separate test runs while
            enabling 0-RTT and early data in the test
            equipment.</t>
          </li>
          <li>
            <t>Application equipment.</t></dd>
          <dt>Application Transactions Per Second</t>
            <t>The Second</dt>
            <dd>The average number of successfully completed transactions per
            second. For a particular transaction to be considered successful,
            all data MUST <bcp14>MUST</bcp14> have been transferred in its
            entirety. In case of an HTTP(S)
            transactions, transaction, it MUST <bcp14>MUST</bcp14>
            have a valid status code (200 OK).</t>
          </li>
          <li>
            <t>TLS OK).</dd>
          <dt>TLS Handshake Rate</t>
            <t>The Rate</dt>
            <dd><t>The average number of successfully established TLS connections
            per second between hosts across the
            DUT/SUT, DUT/SUT or between hosts and
            the DUT/SUT.</t>
            <t>For TLS1.3 TLS 1.3, the handshake rate can be measured with the 0-RTT or
            1-RTT handshake. The transport protocol can be either TCP or QUIC.</t>
          </li>
          <li>
            <t>Inspected Throughput</t>
            <t>The
            QUIC.</t></dd>
          <dt>Inspected Throughput</dt>
            <dd>The number of bits per second of examined and allowed traffic
            a network security device is able to transmit to the correct
            destination interface(s) in response to a specified offered
            load. The throughput benchmarking tests defined in <xref
            target="Benchmarking" format="default"/> SHOULD <bcp14>SHOULD</bcp14>
            measure the average Layer layer 2 throughput value when the DUT/SUT is
            "inspecting" traffic. It is also acceptable to measure other OSI Layer
            layer throughput. However, the measured layer (e.g. Layer (e.g., layer 3
            throughput)
            MUST <bcp14>MUST</bcp14> be noted in the report report, and the
            user MUST <bcp14>MUST</bcp14> be aware of the implication while
            comparing the throughput performance of multiple
            DUT/SUTs DUTs/SUTs measured
            in different OSI Layers. layers. This document recommends presenting the
            inspected throughput value in Gbit/s rounded to two places of
            precision with a more specific Kbit/s kbit/s in
            parenthesis.</t>
          </li>
          <li>
            <t>Time parenthesis.</dd>
          <dt>Time to First Byte (TTFB)</t>
            <t>TTFB is the (TTFB)</dt>
            <dd>The elapsed time between the start of sending the TCP
            SYN packet or QUIC initial Client Hello from the client and the
            client receiving the first packet of application data from the
            server via the DUT/SUT. The benchmarking tests <xref
            target="HTTP-Latency" format="default">HTTP
            Transaction Latency</xref> transaction
            latency</xref> and <xref target="HTTPS-Latency"
            format="default">HTTPS Transaction Latency</xref> transaction latency</xref> measure the
            minimum, average average, and maximum TTFB. The value should be expressed
            in milliseconds.</t>
          </li>
          <li>
            <t>URL milliseconds.</dd>
          <dt>URL Response time Time / Time to Last Byte (TTLB)</t>
            <t>URL
            Response time / TTLB is the (TTLB)</dt>
            <dd>The elapsed time between the start
            of sending the TCP SYN packet or QUIC initial Client Hello from
            the client and the client receiving the last packet of application
            data from the server via the DUT/SUT. The benchmarking tests <xref
            target="HTTP-Latency" format="default">HTTP Transaction
            Latency</xref> transaction
            latency</xref> and <xref target="HTTPS-Latency"
            format="default">HTTPS Transaction Latency</xref> transaction latency</xref> measure the
            minimum, average average, and maximum TTLB. The value should be expressed
            in milliseconds.</t>
          </li>
        </ul> milliseconds.</dd>
        </dl>
      </section>
    </section>
    <section anchor="Benchmarking" numbered="true" toc="default">
      <name>Benchmarking Tests</name>
      <t>This section mainly focuses on the benchmarking tests with HTTP/1.1
      or HTTP/2 traffic traffic, which uses TCP as the transport protocol. In
      particular, this section does not define specific benchmarking tests for
      KPIs related to QUIC or HTTP/3 related KPIs. HTTP/3.
      However, the test methodology defined in
      the benchmarking tests <xref format="default" target="HTTPS_CPS">TCP/QUIC Connections Per Second target="HTTPS_CPS">TCP or QUIC connections per second with HTTPS
      Traffic</xref>,
      traffic</xref>, <xref format="default" target="HTTPS-Latency">HTTPS
      Transaction Latency</xref>,
      transaction latency</xref>, <xref format="default" target="HTTPS_TP">HTTPS Throughput</xref>, throughput</xref>, and <xref format="default" target="HTTPS_CC">Concurrent TCP/QUIC Connection Capacity target="HTTPS_CC">concurrent TCP or QUIC connection capacity with HTTPS
      Traffic </xref>
      traffic</xref> can be used to test KPIs related to QUIC or HTTP/3 related KPIs. HTTP/3. The
      throughput performance test with the application traffic mix defined in
      <xref target="Throughput_Performance_With_Traffic_Mix" format="default"/>
      can be performed with any other application traffic traffic, including
      HTTP/3.</t>
      <section anchor="Throughput_Performance_With_Traffic_Mix" numbered="true" toc="default">
        <name>Throughput Performance with Application Traffic Mix</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Using a relevant application traffic mix, determine the
          sustainable inspected throughput supported by the DUT/SUT.</t>
          <t>Based on the test customer's specific use case, testers can
          choose the relevant application traffic mix for this test. The
          details about the traffic mix MUST <bcp14>MUST</bcp14> be documented in the report. At
          least
          least, the following traffic mix details MUST <bcp14>MUST</bcp14> be documented and
          reported together with the test results:</t>
          <ul empty="true" spacing="normal">
            <li>Name of applications and layer 7 protocols</li>
            <li>Percentage of emulated traffic for each application and layer
              7 protocol</li>
            <li>Percentage of encrypted traffic and used cipher suites and
              keys (The RECOMMENDED (the <bcp14>RECOMMENDED</bcp14> ciphers and keys are defined in <xref target="Emulated_web_Browser_attributes" format="default"/>.)</li> format="default"/>)</li>
            <li>Used object sizes for each application and layer 7
              protocols</li>
          </ul>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any benchmarking test specific benchmarking-test-specific testbed
          configuration changes MUST <bcp14>MUST</bcp14> be documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, the benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented. In case If the DUT/SUT is configured without TLS inspection, the test report
            MUST <bcp14>MUST</bcp14> explain how this impacts the implications encrypted traffic of this to the relevant application traffic mix encrypted traffic.</t> mix.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_TC_7_1" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <ul empty="true" spacing="normal">
              <li>Client IP address ranges defined in <xref target="Client_IP"
              format="default"/></li>
              <li>Server IP address ranges defined in <xref target="Server_IP"
              format="default"/></li>
              <li>Traffic distribution ratio between IPv4 and IPv6 defined in
                <xref target="Client_IP" format="default"/></li>
              <li>Target inspected throughput: Aggregated line rate of the
                interface(s) one or
                more interfaces used in the DUT/SUT or the value defined based on
                the requirement for a specific deployment scenario</li>
              <li>Initial
              <li><t>Initial throughput: 10% of the "Target inspected
                throughput" Note:
              throughput"</t>
              <t>Note: Initial throughput is not a KPI to report.  This value
              is configured on the traffic generator and used to perform Step 1: "Test
              1 (Test Initialization and Qualification" Qualification) described under in <xref
              target="Test_Procedures_and_Expected_Results_TC_7_1" format="default"/>.</li>
              format="default"/>.</t></li>
	      <li>One of the ciphers and keys defined in <xref
              target="Emulated_web_Browser_attributes" format="default"/> are RECOMMENDED is
              <bcp14>RECOMMENDED</bcp14> to use for this benchmarking
              test.</li>
	    </ul>
          </section>
          <section anchor="Traffic_Profile" numbered="true" toc="default">
            <name>Traffic Profile</name>
            <t>Traffic profile: This
            <t>This test MUST <bcp14>MUST</bcp14> be run with a relevant
            application traffic mix profile.</t>
          </section>
          <section anchor="Test_Results_Validation_Criteria_7_1" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions MUST <bcp14>MUST</bcp14> be less than
                0.001% (1 out of 100,000 transactions) of total the attempted
                transactions.</li>
              <li>Number
              <li>The number of Terminated terminated TCP connections due to unexpected TCP
                RST sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections.</li>
              <li>If HTTP/3 is used, the number of failed QUIC connections
                due to unexpected HTTP/3 error codes MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 connections) of the total initiated QUIC
                connections.</li>
            </ol>
          </section>
          <section anchor="Measurement_7_1" numbered="true" toc="default">
            <name>Measurement</name>
            <t>The following KPI metrics MUST <bcp14>MUST</bcp14> be reported for this
            benchmarking test:</t>
            <t>Mandatory
	    <ul spacing="normal">
	      <li><t>Mandatory KPIs (benchmarks): Inspected Throughput inspected throughput and
            Application Transactions Per Second</t>
	      application transactions per second</t>
              <t>Note: The TTLB MUST <bcp14>MUST</bcp14> be reported along
              with the object size used in the traffic profile.</t>
            <t>Optional TCP stack related profile.</t></li>
	      <li>Optional TCP-stack-related KPIs: TCP Connections Per Second, connections per second,
	      TLS Handshake Rate, handshake rate, TTFB (minimum, average, and maximum), TTLB
	      (minimum, average, and maximum)</t>
            <t>Optional QUIC stack related maximum)</li>
	      <li>Optional QUIC-stack-related KPIs: QUIC connection connections per second
	      and concurrent QUIC connections</t> connections</li></ul>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_1" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedures are designed to measure the inspected
          throughput performance of the DUT/SUT at the sustaining period of
          the traffic load profile. The test procedure consists of three major
          steps:
          steps. Step 1 ensures the DUT/SUT is able to reach the performance
          value (initial throughput) and meets the test results validation
          criteria when it was very minimally utilized. Step 2 determines
          whether the DUT/SUT is able to reach the target performance value
          within the test results validation criteria.  Step 3 determines the
          maximum achievable performance value within the test results
          validation criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IP types: IPv4 only, IPv6 only, and IPv4 and IPv6 mixed traffic
          distribution.</t>
          <section anchor="Step1_Test_Initialization" numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            generate test traffic at the "Initial "initial throughput" rate rate, as
            described in <xref target="Test_Equipment_Configuration_Parameters_TC_7_1" format="default"/>. The
            test equipment MUST <bcp14>MUST</bcp14> follow the traffic load profile definition as
            described in <xref target="Traffic_Load_Profile" format="default"/>. The DUT/SUT
            MUST
            <bcp14>MUST</bcp14> reach the "Initial "initial throughput" during the sustain phase.
            Measure all KPI KPIs, as defined in <xref target="Measurement_7_1" format="default"/>. The measured KPIs during the sustain
            phase MUST <bcp14>MUST</bcp14> meet all the test results validation criteria defined
            in <xref target="Test_Results_Validation_Criteria_7_1" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to step Step 2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to generate traffic at the "Target
            inspected throughput" rate defined in <xref target="Test_Equipment_Configuration_Parameters_TC_7_1" format="default"/>. The
            test equipment MUST <bcp14>MUST</bcp14> follow the traffic load profile definition as
            described in <xref target="Traffic_Load_Profile" format="default"/>. The test
            equipment MUST <bcp14>MUST</bcp14> start to measure and record all specified KPIs.
            Continue the test until all traffic profile phases are
            completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective
            ("Target inspected throughput") in the sustain phase. Follow step
            3, Step
            3 if the measured value does not meet the target value or does
            not fulfill the test results validation criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable average inspected throughput within
            the test results validation criteria. The final test iteration
            MUST
            <bcp14>MUST</bcp14> be performed for the test duration defined in <xref target="Traffic_Load_Profile" format="default"/>.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTP_CPS" numbered="true" toc="default">
        <name>TCP/HTTP
        <name>TCP Connections Per Second</name> Second with HTTP Traffic</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Using HTTP traffic, determine the sustainable TCP connection
          establishment rate supported by the DUT/SUT under different
          throughput load conditions.</t>
          <t>To measure connections per second, test iterations MUST <bcp14>MUST</bcp14> use
          different fixed HTTP response object sizes (the different load
          conditions) defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_CPS" format="default"/>.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTP_CPS" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14>
            conform to the requirements defined in <xref
            target="Test_Equipment_Configuration" format="default"/>. The
            following parameters
            MUST <bcp14>MUST</bcp14> be documented for this
            benchmarking test:</t>
            <t>Client
	    <ul spacing="normal">
	      <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></t>
            <t>Server
	      format="default"/></li>
	      <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></t>
            <t>Traffic
	      format="default"/></li>
	      <li>Traffic distribution ratio between IPv4 and IPv6 defined in
	      <xref target="Client_IP" format="default"/></t>
            <t>Target format="default"/></li>
	      <li>Target connections per second: Initial value from the product
	      datasheet or the value defined based on the requirement for a
	      specific deployment scenario</t>
            <t>Initial scenario</li>
	      <li><t>Initial connections per second: 10% of "Target connections
	      per
            second" (Note: second"</t>
	      <t>Note: Initial connections per second is not a KPI
	      to report. This value is configured on the traffic generator and
	      used to perform Step1: "Test Step 1 (Test Initialization and Qualification" Qualification)
	      described under in <xref
	      target="Test_Procedures_and_Expected_Results_TC_7_2" format="default"/>.)</t>
	      format="default"/>.</t></li>
	      <li>The <bcp14>RECOMMENDED</bcp14> response object sizes are 1,
	      2, 4, 16, and 64 KB.</li>
	    </ul>
	      <t>The client MUST <bcp14>MUST</bcp14> negotiate HTTP and close the
	      connection with FIN immediately after the completion of one
	      transaction. In each test iteration, the client MUST
	      <bcp14>MUST</bcp14> send a GET request requesting a fixed HTTP
	      response object size.</t>
            <t>The RECOMMENDED response object sizes are 1, 2, 4, 16, and 64
            KByte.</t>
          </section>
          <section anchor="Validation_Criteria_HTTP_CPS" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The Test test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of total the attempted
                transactions.</li>
              <li>Number
              <li>The number of terminated TCP connections due to unexpected TCP
                RST sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections.</li>
              <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
                constant rate (considered (it is considered as a constant rate if any deviation
                of the traffic forwarding rate is less than 5%).</li>
              <li>Concurrent TCP connections MUST <bcp14>MUST</bcp14> be constant during steady
                state
                state, and any deviation of concurrent TCP connections MUST <bcp14>MUST</bcp14> be
                less than 10%. This confirms the DUT opens and closes TCP
                connections at approximately the same rate.</li>
            </ol>
          </section>
          <section numbered="true" toc="default">
            <name>Measurement</name>
            <t>TCP Connections Per Second MUST connections per second <bcp14>MUST</bcp14> be reported for each test
            iteration (for each object size).</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_2" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The
          <t>   The test procedure is designed to measure the DUT/SUT's rate of TCP
   connections per second rate of the DUT/SUT at during the sustaining period of the traffic
   load profile. The test procedure consists of three major steps: steps. Step
          1 ensures the DUT/SUT is able to reach the performance value
          (Initial connections per second) and meets the test results
          validation criteria when it was very minimally utilized.  Step 2
          determines whether the DUT/SUT is able to reach the target
          performance value within the test results validation criteria.  Step
          3 determines the maximum achievable performance value within the
          test results validation criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IP types: IPv4 only, IPv6 only, and IPv4 and IPv6 mixed traffic
          distribution.</t>
          <section numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            establish "Initial connections per second" second", as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_CPS" format="default"/>. The
            traffic load profile MUST <bcp14>MUST</bcp14> be defined as described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>The DUT/SUT MUST <bcp14>MUST</bcp14> reach the "Initial connections per second"
            before the sustain phase. The measured KPIs during the sustain
            phase MUST <bcp14>MUST</bcp14> meet all the test results validation criteria defined
            in <xref target="Validation_Criteria_HTTP_CPS" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> continue to "Step 2".</t> Step 2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the target objective
            ("Target connections per second") defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_CPS" format="default"/>. The
            test equipment MUST <bcp14>MUST</bcp14> follow the traffic load profile definition as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>During the ramp up and sustain phase phases of each test iteration,
            other KPIs KPIs, such as inspected throughput, concurrent TCP
            connections, and application transactions per second MUST NOT second, <bcp14>MUST NOT</bcp14>
            reach the maximum value the DUT/SUT can support. The test results
            for specific test iterations MUST NOT <bcp14>MUST NOT</bcp14> be reported as valid results
            if the above mentioned abovementioned KPI (especially inspected throughput)
            reaches the maximum value. (Example: If (For example, if the test iteration with 64
            KByte
            KB of HTTP response object size reached the maximum inspected
            throughput limitation of the DUT/SUT, the test iteration MAY <bcp14>MAY</bcp14> be
            interrupted and the result for 64 KByte KB must not be reported.)</t>
            <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record all
            specified KPIs. Continue the test until all traffic profile phases
            are completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective
            ("Target connections per second") in the sustain phase. Follow
            step 3,
            Step 3 if the measured value does not meet the target value or
            does not fulfill the test results validation criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable TCP connections per second within the
            test results validation criteria.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTP_TP" numbered="true" toc="default">
        <name>HTTP Throughput</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Determine the sustainable inspected throughput of the DUT/SUT for
          HTTP transactions varying the HTTP response object size.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTP_TP" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <t>Client
	    <ul>
	      <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></t>
            <t>Server
	      format="default"/></li>
	      <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></t>
            <t>Traffic
	      format="default"/></li>
	      <li>Traffic distribution ratio between IPv4 and IPv6 defined in
	      <xref target="Client_IP" format="default"/></t>
            <t>Target format="default"/></li>
	      <li>Target inspected throughput: Aggregated line rate of the
            interface(s) one
	      or more interfaces used in the DUT/SUT or the value defined based on
	      the requirement for a specific deployment scenario</t>
            <t>Initial scenario</li>
	      <li><t>Initial throughput: 10% of "Target inspected throughput" Note: throughput"</t>
	      <t>Note: Initial throughput is not a KPI to report. This value is
	      configured on the traffic generator and used to perform Step 1:
            "Test 1
	      (Test Initialization and Qualification" Qualification) described under in <xref
	      target="Test_Procedures_and_Expected_Results_TC_7_3" format="default"/>.</t>
            <t>Number
	      format="default"/>.</t></li>
	      <li>Number of HTTP response object requests (transactions) per
	      connection: 10</t>
            <t>RECOMMENDED 10</li>
	      <li><bcp14>RECOMMENDED</bcp14> HTTP response object size: 1, 16,
	      64, and 256 KByte, KB and mixed objects defined in <xref
	      target="table4" format="default"/>.</t> format="default"/></li>
	    </ul>
            <table anchor="table4" align="center">
              <name>Mixed Objects</name>
              <thead>
                <tr>
                  <th align="left">Object size (KByte)</th> (KB)</th>
                  <th align="left">Number of requests/ requests / Weight</th>
                </tr>
              </thead>
              <tbody>
                <tr>
                  <td align="left">0.2</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">6</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">8</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">9</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">10</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">25</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">26</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">35</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">59</td>
                  <td align="left">1</td>
                </tr>
                <tr>
                  <td align="left">347</td>
                  <td align="left">1</td>
                </tr>
              </tbody>
            </table>
          </section>
          <section anchor="Validation_Criteria_HTTP_TP" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of attempt the total attempted transactions.</li>
              <li>Traffic MUST <bcp14>MUST</bcp14> be forwarded at a constant rate (considered (it is considered as
                a constant rate if any deviation of the traffic forwarding rate is
                less than 5%).</li>
              <li>Concurrent TCP connections MUST <bcp14>MUST</bcp14> be constant during steady
                state
                state, and any deviation of concurrent TCP connections MUST <bcp14>MUST</bcp14> be
                less than 10%. This confirms the DUT opens and closes TCP
                connections at approximately the same rate.</li>
            </ol>
          </section>
          <section anchor="Measurement_TP" numbered="true" toc="default">
            <name>Measurement</name>
            <t>Inspected Throughput throughput and HTTP Transactions transactions per Second MUST second <bcp14>MUST</bcp14> be
            reported for each object size.</t>
            <t/>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_3" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedure is designed to measure HTTP throughput of the
          DUT/ SUT. The test procedure consists of three major steps: steps. Step 1
          ensures the DUT/SUT is able to reach the performance value (Initial (initial
          throughput) and meets the test results validation criteria when it
          was very minimal minimally utilized.  Step 2 determines whether the DUT/SUT is
          able to reach the target performance value within the test results
          validation criteria.  Step 3 determines the maximum achievable
          performance value within the test results validation criteria.</t>
	    <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple
	    times with different IPv4 and IPv6 traffic distribution distributions and HTTP
	    response object sizes.</t>
          <section numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            establish "Initial inspected throughput" "initial throughput", as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_TP" format="default"/>.</t>
            <t>The traffic load profile MUST <bcp14>MUST</bcp14> be defined as described in <xref target="Traffic_Load_Profile" format="default"/>. The DUT/SUT MUST <bcp14>MUST</bcp14> reach the
            "Initial inspected
            "initial throughput" during the sustain phase. Measure
            all KPI KPIs, as defined in <xref target="Measurement_TP" format="default"/>.</t>
            <t>The measured KPIs during the sustain phase MUST <bcp14>MUST</bcp14> meet the test
            results validation criteria "a" defined in <xref target="Validation_Criteria_HTTP_TP" format="default"/>. The test results
            validation criteria "b" and "c" are OPTIONAL <bcp14>OPTIONAL</bcp14> for step Step 1.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the target objective
            ("Target inspected throughput") defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_TP" format="default"/>. The
            test equipment MUST <bcp14>MUST</bcp14> start to measure and record all specified
            KPIs. Continue the test until all traffic profile phases are
            completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective in the
            sustain phase. Follow step 3, Step 3 if the measured value does not meet
            the target value or does not fulfill the test results validation
            criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable inspected throughput within the test
            results validation criteria and measure the KPI metric
            Transactions
            transactions per Second. second. The final test iteration MUST <bcp14>MUST</bcp14> be
            performed for the test duration defined in <xref target="Traffic_Load_Profile" format="default"/>.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTP-Latency" numbered="true" toc="default">
        <name>HTTP Transaction Latency</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Using HTTP traffic, determine the HTTP transaction latency when the
          DUT is running with sustainable HTTP transactions per second
          supported by the DUT/SUT under different HTTP response object
          sizes.</t>
          <t>Test iterations MUST <bcp14>MUST</bcp14> be performed with different HTTP response
          object sizes in two different scenarios. One scenarios: one with a single
          transaction and the other with multiple transactions within a single
          TCP connection. For consistency, both the single and multiple
          transaction tests MUST <bcp14>MUST</bcp14> be configured with the same HTTP version</t> version.</t>
          <t>Scenario 1: The client MUST <bcp14>MUST</bcp14> negotiate HTTP and close the
          connection with FIN immediately after the completion of a single
          transaction (GET and RESPONSE).</t>
          <t>Scenario 2: The client MUST <bcp14>MUST</bcp14> negotiate HTTP and close the
          connection with FIN immediately after the completion of 10 transactions
          (GET and RESPONSE) within a single TCP connection.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTP_latency" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <t>Client
	    <ul spacing="normal">
	      <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></t>
            <t>Server
	      format="default"/></li>
	      <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></t>
            <t>Traffic
	      format="default"/></li>
	      <li>Traffic distribution ratio between IPv4 and IPv6 defined in
	      <xref target="Client_IP" format="default"/></t>
            <t/>
            <t>Target format="default"/></li>
	      <li>Target objective for scenario 1: 50% of the connections per
	      second measured in the benchmarking test <xref format="default" target="HTTP_CPS">TCP/HTTP Connections Per Second</xref></t>
            <t>Target
	      target="HTTP_CPS">TCP connections per second with HTTP traffic</xref></li>
	      <li>Target objective for scenario 2: 50% of the inspected
	      throughput measured in the benchmarking test <xref format="default"
	      target="HTTP_TP">HTTP Throughput</xref></t>
            <t>Initial throughput</xref></li>
	      <li>Initial objective for scenario 1: 10% of "Target objective
	      for scenario 1"</t>
            <t>Initial 1"</li>
	      <li><t>Initial objective for scenario 2: 10% of "Target objective
	      for scenario 2"</t>
	      <t>Note: The Initial initial objectives are not a KPI KPIs to
	      report. These values are configured on the traffic generator and
	      used to perform
            Step1: "Test Step 1 (Test Initialization and Qualification" Qualification)
	      described under in <xref
	      target="Test_Procedures_and_Expected_Results_TC_7_4" format="default"/>.</t>
            <t>HTTP
	      format="default"/>.</t></li>
	      <li>HTTP transaction per TCP connection: Test scenario 1 with a
	      single transaction and test scenario 2 with 10 transactions.</t>
            <t>HTTP transactions</li>
	      <li>HTTP with GET request requesting a single object. object: The
            RECOMMENDED
	      <bcp14>RECOMMENDED</bcp14> object sizes are 1, 16, and 64 KByte.
	      KB. For each test iteration, the client MUST <bcp14>MUST</bcp14>
	      request a single HTTP response object
            size.</t> size.</li>
	    </ul>
          </section>
          <section anchor="Validation_Criteria_HTTP_Latency" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The Test test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of attempt the total attempted transactions.</li>
              <li>Number
              <li>The number of terminated TCP connections due to unexpected TCP
                RST sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections.</li>
              <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
                constant rate (considered (it is considered as a constant rate if any deviation
                of the traffic forwarding rate is less than 5%).</li>
              <li>Concurrent TCP connections MUST <bcp14>MUST</bcp14> be constant during steady
                state
                state, and any deviation of concurrent TCP connections MUST <bcp14>MUST</bcp14> be
                less than 10%. This confirms the DUT opens and closes TCP
                connections at approximately the same rate.</li>
              <li>After ramp up up, the DUT MUST <bcp14>MUST</bcp14> achieve the "Target objective" target objectives
                defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_latency" format="default"/>
                and remain in that state for the entire test duration (sustain
                phase).</li>
            </ol>
          </section>
          <section numbered="true" toc="default">
            <name>Measurement</name>
            <t>TTFB
            <t>The TTFB (minimum, average, and maximum) and TTLB (minimum,
            average, and maximum) MUST <bcp14>MUST</bcp14> be reported for each object size.</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_4" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedure is designed to measure the TTFB or TTLB when the
          DUT/SUT is operating close to 50% of its maximum achievable
          connections per second or inspected throughput. The test procedure
          consists of two major steps: steps. Step 1 ensures the DUT/SUT is able to
          reach the initial performance values and meets the test results
          validation criteria when it was very minimally utilized.  Step 2
          measures the latency values within the test results validation
          criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times
          with different IP types (IPv4 only, IPv6 only, and IPv4 and IPv6
          mixed traffic distribution), HTTP response object sizes, and single
          and multiple transactions per connection scenarios.</t>
          <section numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            establish the "Initial objective" initial objectives, as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_latency" format="default"/>.
            The traffic load profile MUST <bcp14>MUST</bcp14> be defined as described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>The DUT/SUT MUST <bcp14>MUST</bcp14> reach the "Initial objective" initial objectives before the
            sustain phase. The measured KPIs during the sustain phase MUST <bcp14>MUST</bcp14>
            meet all the test results validation criteria defined in <xref target="Validation_Criteria_HTTP_Latency" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the "Target objective" target objectives
            defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_latency" format="default"/>.
            The test equipment MUST <bcp14>MUST</bcp14> follow the traffic load profile definition
            as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record all
            specified KPIs. Continue the test until all traffic profile phases
            are completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT MUST <bcp14>MUST</bcp14>
            reach the desired value of the target objective in the sustain
            phase.</t>
            <t>Measure the minimum, average, and maximum values of the TTFB and
            TTLB.</t>
          </section>
        </section>
      </section>
      <section numbered="true" toc="default">
        <name>Concurrent TCP/HTTP TCP Connection Capacity</name> Capacity with HTTP Traffic</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Determine the number of concurrent TCP connections that the DUT/
          SUT DUT/SUT sustains when
	  using HTTP traffic.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section anchor="CC_parameter" numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTP_CC" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be noted for this benchmarking test:</t>
            <ul empty="true" spacing="normal">
              <li>Client IP address ranges defined in <xref target="Client_IP"
              format="default"/></li>
              <li>Server IP address ranges defined in <xref target="Server_IP"
              format="default"/></li>
              <li>Traffic distribution ratio between IPv4 and IPv6 defined in
                <xref target="Client_IP" format="default"/></li>
              <li>Target concurrent connection: Initial value from the product
                datasheet or the value defined based on the requirement for a
                specific deployment scenario.</li>
              <li>Initial scenario</li>
              <li><t>Initial concurrent connection: 10% of "Target concurrent
                connection" Note:
                connection"</t>
		<t>Note: Initial concurrent connection is not a KPI to
		report. This value is configured on the traffic generator and
		used to perform Step1: "Test Step 1 (Test Initialization and
                Qualification" Qualification)
		described under in <xref
		target="Test_Procedures_and_Expected_Results_TC_7_5" format="default"/>.</li>
		format="default"/>.</t></li>
              <li>Maximum connections per second during ramp up phase: 50% of
                maximum connections per second measured in the benchmarking test
                <xref target="HTTP_CPS" format="default">TCP/HTTP Connections format="default">TCP connections per
                second</xref></li>
                second with HTTP traffic</xref></li>
              <li>Ramp up time (in traffic load profile for "Target
                concurrent connection"): "Target concurrent connection" /
                "Maximum connections per second during ramp up phase"</li>
              <li>Ramp up time (in traffic load profile for "Initial
                concurrent connection"): "Initial concurrent connection" /
                "Maximum connections per second during ramp up phase"</li>
            </ul> phase"</li></ul>
		<t>The client MUST <bcp14>MUST</bcp14> negotiate HTTP HTTP, and each
		client MAY <bcp14>MAY</bcp14> open multiple concurrent TCP
		connections per server endpoint IP.</t>
		<t>Each client sends 10 GET requests requesting 1 KByte KB HTTP
		response object in the same TCP connection (10 transactions/TCP
            connection)
		transactions / TCP connections), and the delay (think time)
		between each transaction
            MUST <bcp14>MUST</bcp14> be X seconds.</t>
            <t>X seconds, where X is as follows.</t>
		<t indent="3">X = ("Ramp up time" + "steady state time") /10</t> / 10</t>
            <t>The established connections MUST <bcp14>MUST</bcp14> remain open until the ramp
            down phase of the test. During the ramp down phase, all
            connections MUST <bcp14>MUST</bcp14> be successfully closed with FIN.</t>
          </section>
          <section anchor="CC_Test_Results_Validation_Criteria" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The Test test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of the total attempted
                transactions.</li>
              <li>Number
              <li>The number of terminated TCP connections due to unexpected TCP
                RST sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections.</li>
              <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
                constant rate (considered (it is considered as a constant rate if any deviation
                of the traffic forwarding rate is less than 5%).</li>
            </ol>
          </section>
          <section anchor="CC_Measurement" numbered="true" toc="default">
            <name>Measurement</name>
            <t>Average Concurrent concurrent TCP Connections MUST connections <bcp14>MUST</bcp14> be reported for this
            benchmarking test.</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_5" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedure is designed to measure the concurrent TCP
          connection capacity of the DUT/SUT at the sustaining period of the
          traffic load profile. The test procedure consists of three major
          steps:
          steps. Step 1 ensures the DUT/SUT is able to reach the performance
          value (Initial concurrent connection) and meets the test results
          validation criteria when it was very minimally utilized.  Step 2
          determines whether the DUT/SUT is able to reach the target
          performance value within the test results validation criteria.  Step
          3 determines the maximum achievable performance value within the
          test results validation criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IPv4 and IPv6 traffic distributions.</t>
          <section anchor="CC_Step1_Test_Initialization" numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure test equipment to establish "Initial concurrent TCP
            connections" defined in <xref target="Test_Equipment_Configuration_Parameters_HTTP_CC" format="default"/>. Except
            ramp up time, the traffic load profile MUST <bcp14>MUST</bcp14> be defined as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>During the sustain phase, the DUT/SUT MUST <bcp14>MUST</bcp14> reach the "Initial
            concurrent TCP connections". The measured KPIs during the sustain
            phase MUST <bcp14>MUST</bcp14> meet all the test results validation criteria defined
            in <xref target="CC_Test_Results_Validation_Criteria" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the target objective
            ("Target concurrent TCP connections"). The test equipment MUST <bcp14>MUST</bcp14>
            follow the traffic load profile definition (except ramp up time)
            as described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>During the ramp up and sustain phase, phases, the other KPIs KPIs, such as
            inspected throughput, TCP connections per second, and application
            transactions per second MUST NOT second, <bcp14>MUST NOT</bcp14> reach the maximum value the
            DUT/SUT can support.</t>
            <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record KPIs
            defined in <xref target="CC_Measurement" format="default"/>. Continue the test
            until all traffic profile phases are completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective in the
            sustain phase. Follow step 3, Step 3 if the measured value does not meet
            the target value or does not fulfill the test results validation
            criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable concurrent TCP connections capacity
            within the test results validation criteria.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTPS_CPS" numbered="true" toc="default">
        <name>TCP/QUIC
        <name>TCP or QUIC Connections per Second with HTTPS Traffic</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Using HTTPS traffic, determine the sustainable TLS session
          establishment rate supported by the DUT/SUT under different
          throughput load conditions.</t>
          <t>Test iterations MUST <bcp14>MUST</bcp14> include common cipher suites and key
          strengths
          strengths, as well as forward looking forward-looking stronger keys. Specific test
          iterations MUST <bcp14>MUST</bcp14> include ciphers and keys defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_CPS" format="default"/>.</t>
          <t>For each cipher suite and key strengths, strength, test iterations MUST <bcp14>MUST</bcp14> use
          a single HTTPS response object size defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_CPS" format="default"/> to
          measure connections per second performance under a variety of
          DUT/SUT security inspection load conditions.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTPS_CPS" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <t>Client
	    <ul spacing="normal">
	      <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></t>
            <t>Server
	      format="default"/></li>
	      <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></t>
            <t>Traffic
	      format="default"/></li>
	      <li>Traffic distribution ratio between IPv4 and IPv6 defined in
	      <xref target="Client_IP" format="default"/></t>
            <t>Target format="default"/></li>
	      <li>Target connections per second: Initial value from the product
	      datasheet or the value defined based on the requirement for a
	      specific deployment scenario.</t>
            <t>Initial scenario</li>
	      <li><t>Initial connections per second: 10% of "Target connections
	      per
            second" (Note: second"</t>
	      <t>Note: Initial connections per second is not a KPI
	      to report. This value is configured on the traffic generator and
	      used to perform Step1: "Test Step 1 (Test Initialization and Qualification" Qualification)
	      described under in <xref
	      target="Test_Procedures_and_Expected_Results_TC_7_6" format="default"/>.)</t>
            <t>RECOMMENDED
	      format="default"/>.)</t></li>
	      <li><bcp14>RECOMMENDED</bcp14> ciphers and keys defined in <xref
	      target="Emulated_web_Browser_attributes" format="default"/></t> format="default"/></li>
	      <li>The <bcp14>RECOMMENDED</bcp14>
            object sizes are 1, 2, 4, 16, and 64 KB.</li>
	    </ul>
            <t>The client MUST <bcp14>MUST</bcp14> negotiate HTTPS and close the connection
            without error immediately after the completion of one transaction.
            In each test iteration, the client MUST <bcp14>MUST</bcp14> send a GET request
            requesting a fixed HTTPS response object size. The RECOMMENDED
            object sizes are 1, 2, 4, 16, and 64 KByte.</t> size.</t>
          </section>
          <section anchor="Validation_Criteria_HTTPS_CPS" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole test duration.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of attempt the attempted transactions.</li>
              <li>Number
              <li>The number of terminated TCP connections due to unexpected TCP
                RST sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections. If HTTP/3 is
                used, the number of terminated QUIC connections due to
                unexpected errors MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated QUIC connections.</li>
              <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
                constant rate (considered (it is considered as a constant rate if any deviation
                of the traffic forwarding rate is less than 5%).</li>
              <li>Concurrent
              <li>The concurrent TCP connections generation rate MUST <bcp14>MUST</bcp14> be constant
                during steady state state, and any deviation of concurrent TCP
                connections MUST <bcp14>MUST</bcp14> be less than 10%. If HTTP/3 is used, the
                concurrent QUIC connections generation rate MUST <bcp14>MUST</bcp14> be constant
                during steady state state, and any deviation of concurrent QUIC
                connections MUST <bcp14>MUST</bcp14> be less than 10%. This confirms the DUT opens
                and closes connections at approximately the same rate.</li>
            </ol>
          </section>
          <section numbered="true" toc="default">
            <name>Measurement</name>
            <t>If HTTP 1.1 or HTTP/2 is used, TCP connections per second MUST <bcp14>MUST</bcp14>
            be reported for each test iteration (for each object size).</t>
            <t>If HTTP/3 is used, QUIC connections per second MUST <bcp14>MUST</bcp14> be measured
            and reported for each test iteration (for each object size).</t>
            <t>The KPI metric TLS Handshake Rate handshake rate can be measured in the test
            using 1 KByte KB object size.</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_6" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The
          <t>   The test procedure is designed to measure the DUT/SUT's rate of TCP
   or QUIC connections per second rate of the DUT/SUT at during the sustaining period of the
   traffic load profile. The test procedure consists of three
          major steps: steps. Step 1 ensures the DUT/SUT is able to reach the
          performance value (Initial connections per second) and meets the
          test results validation criteria when it was very minimally
          utilized.  Step 2 determines whether the DUT/SUT is able to reach
          the target performance value within the test results validation
          criteria.  Step 3 determines the maximum achievable performance
          value within the test results validation criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IPv4 and IPv6 traffic distributions.</t>
          <section anchor="TLS_Handshake_Step1_Test_Initialization" numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            establish "Initial connections per second" second", as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_CPS" format="default"/>. The
            traffic load profile MUST <bcp14>MUST</bcp14> be defined as described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>The DUT/SUT MUST <bcp14>MUST</bcp14> reach the "Initial connections per second"
            before the sustain phase. The measured KPIs during the sustain
            phase MUST <bcp14>MUST</bcp14> meet all the test results validation criteria defined
            in <xref target="Validation_Criteria_HTTPS_CPS" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish "Target connections per
            second"
            second", as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_CPS" format="default"/>. The
            test equipment MUST <bcp14>MUST</bcp14> follow the traffic load profile definition as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>During the ramp up and sustain phase, phases, other KPIs KPIs, such as
            inspected throughput, concurrent TCP/QUIC TCP or QUIC connections, and
            application transactions per second MUST NOT second, <bcp14>MUST NOT</bcp14> reach the maximum
            value the DUT/SUT can support. The test results for the specific
            test iteration MUST NOT <bcp14>MUST NOT</bcp14> be reported as valid results, results if the above
            mentioned
	    abovementioned KPI (especially inspected throughput) reaches the
            maximum value. (Example: If (For example, if the test iteration with 64 KByte KB of
            HTTPS response object size reached the maximum inspected
            throughput limitation of the DUT, the test iteration MAY <bcp14>MAY</bcp14> be
            interrupted, and the result for 64 KByte KB should not be
            reported).</t>
            <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record all
            specified KPIs. Continue the test until all traffic profile phases
            are completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective
            ("Target connections per second") in the sustain phase. Follow
            step 3,
            Step 3 if the measured value does not meet the target value or
            does not fulfill the test results validation criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable connections per second within the test
            results validation criteria.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTPS_TP" numbered="true" toc="default">
        <name>HTTPS Throughput</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Determine the sustainable inspected throughput of the DUT/SUT for
          HTTPS transactions by varying the HTTPS response object size.</t>
          <t>Test iterations MUST <bcp14>MUST</bcp14> include common cipher suites and key
          strengths
          strengths, as well as forward looking forward-looking stronger keys. Specific test
          iterations MUST <bcp14>MUST</bcp14> include the ciphers and keys defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_TP" format="default"/>.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTPS_TP" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <t>Client
	    <ul spacing="normal">
	      <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></t>
            <t>Server
	      format="default"/></li>
	      <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></t>
            <t>Traffic
	      format="default"/></li>
	      <li>Traffic distribution ratio between IPv4 and IPv6 defined in
	      <xref target="Client_IP" format="default"/></t>
            <t>Target format="default"/></li>
	      <li>Target inspected throughput: Aggregated line rate of the
            interface(s) one or more
	      interfaces used in the DUT/SUT or the value defined based on
	      the requirement for a specific deployment scenario.</t>
            <t>Initial scenario</li>
	      <li><t>Initial throughput: 10% of "Target inspected throughput" Note: throughput"</t>
	      <t>Note: Initial throughput is not a KPI to report. This value is
	      configured on the traffic generator and used to perform Step1:
            "Test Step 1
	      (Test Initialization and Qualification" Qualification) described under in <xref
	      target="Test_Procedures_and_Expected_Results_TC_7_7" format="default"/>.</t>
            <t>Number
	      format="default"/>.</t></li>
	      <li>Number of HTTPS response object requests (transactions) per
	      connection: 10</t>
            <t>RECOMMENDED 10</li>
	      <li><bcp14>RECOMMENDED</bcp14> ciphers and keys defined in <xref
	      target="Emulated_web_Browser_attributes" format="default"/></t>
            <t>RECOMMENDED format="default"/></li>
	      <li><bcp14>RECOMMENDED</bcp14> HTTPS response object size: 1,
	      16, 64, and 256 KByte, KB and mixed objects defined in <xref
	      target="table4" format="default"/> under of <xref
	      target="Test_Equipment_Configuration_Parameters_HTTP_TP" format="default"/>.</t>
	      format="default"/></li>
	    </ul>
          </section>
          <section anchor="Validation_Criteria_HTTPS_TP" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed Application application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of attempt the attempted transactions.</li>
              <li>Traffic MUST <bcp14>MUST</bcp14> be generated at a constant rate (considered (it is considered as
                a constant rate if any deviation of the traffic forwarding rate is
                less than 5%).</li>
              <li>Concurrent
              <li>The concurrent generated TCP connections MUST <bcp14>MUST</bcp14> be constant
                during steady state state, and any deviation of concurrent TCP
                connections MUST <bcp14>MUST</bcp14> be less than 10%. If HTTP/3 is used, the
                concurrent generated QUIC connections MUST <bcp14>MUST</bcp14> be constant during
                steady state state, and any deviation of concurrent QUIC connections
                MUST
                <bcp14>MUST</bcp14> be less than 10%. This confirms the DUT opens and closes
                connections at approximately the same rate.</li>
            </ol>
          </section>
          <section anchor="Measurement_HTTPS_TP" numbered="true" toc="default">
            <name>Measurement</name>
            <t>Inspected Throughput throughput and HTTPS Transactions transactions per Second MUST second <bcp14>MUST</bcp14> be
            reported for each object size.</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_7" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedure consists of three major steps: steps. Step 1 ensures
          the DUT/SUT is able to reach the performance value (Initial (initial
          throughput) and meets the test results validation criteria when it
          was very minimally utilized.  Step 2 determines whether the DUT/SUT
          is able to reach the target performance value within the test
          results validation criteria.  Step 3 determines the maximum
          achievable performance value within the test results validation
          criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IPv4 and IPv6 traffic distribution distributions and HTTPS response object
          sizes.</t>
          <section numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            establish "Initial throughput" "initial throughput", as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_TP" format="default"/>.</t>
            <t>The traffic load profile MUST <bcp14>MUST</bcp14> be defined as described in <xref target="Traffic_Load_Profile" format="default"/>. The DUT/SUT MUST <bcp14>MUST</bcp14> reach the
            "Initial
            "initial throughput" during the sustain phase. Measure all KPI KPIs, as
            defined in <xref target="Measurement_HTTPS_TP" format="default"/>.</t>
            <t>The measured KPIs during the sustain phase MUST <bcp14>MUST</bcp14> meet the test
            results validation criteria "a" defined in <xref target="Validation_Criteria_HTTPS_TP" format="default"/>. The test results
            validation criteria "b", "b" and "c" are OPTIONAL <bcp14>OPTIONAL</bcp14> for step Step 1.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the target objective
            ("Target inspected throughput") defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_TP" format="default"/>. The
            test equipment MUST <bcp14>MUST</bcp14> start to measure and record all specified
            KPIs. Continue the test until all traffic profile phases are
            completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective in the
            sustain phase. Follow step 3, Step 3 if the measured value does not meet
            the target value or does not fulfill the test results validation
            criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable average inspected throughput within
            the test results validation criteria. The final test iteration
            MUST
            <bcp14>MUST</bcp14> be performed for the test duration defined in <xref target="Traffic_Load_Profile" format="default"/>.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTPS-Latency" numbered="true" toc="default">
        <name>HTTPS Transaction Latency</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Using HTTPS traffic, determine the HTTPS transaction latency when
          the DUT/SUT is running with sustainable HTTPS transactions per second
          supported by the DUT/SUT under different HTTPS response object
          sizes.</t>
          <t>Scenario 1: The client MUST <bcp14>MUST</bcp14> negotiate HTTPS and close the
          connection immediately after the completion of a single transaction
          (GET and RESPONSE).</t>
          <t>Scenario 2: The client MUST <bcp14>MUST</bcp14> negotiate HTTPS and close the
          connection immediately after the completion of 10 transactions (GET
          and RESPONSE) within a single TCP or QUIC connection.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTPS_Latency" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <t>Client
	    <ul spacing="normal">
	      <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></t>
            <t>Server
	      format="default"/></li>
	      <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></t>
            <t>Traffic
	      format="default"/></li>
	      <li>Traffic distribution ratio between IPv4 and IPv6 defined in
	      <xref target="Client_IP" format="default"/></t>
            <t>RECOMMENDED format="default"/></li>
	      <li><bcp14>RECOMMENDED</bcp14> cipher suites and key sizes
	      defined in <xref target="Emulated_web_Browser_attributes" format="default"/></t>
            <t>Target
	      format="default"/></li>
	      <li>Target objective for scenario 1: 50% of the connections per
	      second measured in the benchmarking test <xref target="HTTPS_CPS" format="default">TCP/QUIC Connections
	      format="default">TCP or QUIC connections per Second second with HTTPS
            Traffic</xref></t>
            <t>Target
	      traffic</xref></li>
	      <li>Target objective for scenario 2: 50% of the inspected
	      throughput measured in the benchmarking test <xref format="default"
	      target="HTTPS_TP">HTTPS Throughput</xref></t>
            <t>Initial throughput</xref></li>
	      <li>Initial objective for scenario 1: 10% of "Target objective
	      for scenario 1"</t>
            <t>Initial 1"</li>
	      <li><t>Initial objective for scenario 2: 10% of "Target objective
	      for scenario 2"</t>
	      <t>Note: The Initial initial objectives are not a KPI KPIs to
	      report. These values are configured on the traffic generator and
	      used to perform
            Step1: "Test Step 1 (Test Initialization and Qualification" Qualification)
	      described under in <xref
	      target="Test_Procedures_and_Expected_Results_TC_7_8" format="default"/>.</t>
            <t>HTTPS
	      format="default"/>.</t></li>
	      <li>HTTPS transaction per TCP or QUIC connection: Test scenario
	      1 with a single transaction and scenario 2 with 10 transactions</t>
            <t>HTTPS
	      transactions</li>
	      <li>HTTPS with GET request requesting a single object. object: The
            RECOMMENDED
	      <bcp14>RECOMMENDED</bcp14> object sizes are 1, 16, and 64 KByte.
	      KB. For each test iteration, the client MUST <bcp14>MUST</bcp14>
	      request a single HTTPS response object
            size.</t> size.</li>
	    </ul>
          </section>
          <section anchor="Validation_Criteria_HTTPS_Latency" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The Test test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of attempt the total attempted transactions.</li>
              <li>Number
              <li>The number of terminated TCP connections due to unexpected TCP
                RST sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections. If HTTP/3 is
                used, the number of terminated QUIC connections due to
                unexpected errors MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated QUIC connections.</li>
              <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
                constant rate (considered (it is considered as a constant rate if any deviation
                of the traffic forwarding rate is less than 5%).</li>
              <li>Concurrent TCP or QUIC connections MUST <bcp14>MUST</bcp14> be constant during
                steady state state, and any deviation of concurrent TCP connections
                MUST
                <bcp14>MUST</bcp14> be less than 10%. If HTTP/3 is used, the concurrent
                generated QUIC connections MUST <bcp14>MUST</bcp14> be constant during steady
                state
                state, and any deviation of concurrent QUIC connections MUST <bcp14>MUST</bcp14> be
                less than 10%. This confirms the DUT opens and closes
                connections at approximately the same rate.</li>
              <li>After ramp up up, the DUT/SUT MUST <bcp14>MUST</bcp14> achieve the "Target
                objective" target
                objectives defined in the parameter parameters in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_Latency" format="default"/>
                and remain in that state for the entire test duration (sustain
                phase).</li>
            </ol>
          </section>
          <section numbered="true" toc="default">
            <name>Measurement</name>
            <t>TTFB
            <t>The TTFB (minimum, average, and maximum) and TTLB (minimum,
            average, and maximum) MUST <bcp14>MUST</bcp14> be reported for each object size.</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_8" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedure is designed to measure the TTFB or TTLB when the
          DUT/SUT is operating close to 50% of its maximum achievable
          connections per second or inspected throughput. The test procedure
          consists of two major steps: steps. Step 1 ensures the DUT/SUT is able to
          reach the initial performance values and meets the test results
          validation criteria when it was is very minimally utilized.  Step 2
          measures the latency values within the test results validation
          criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IP types (IPv4 only, IPv6 only, and IPv4 and IPv6 mixed traffic
          distribution), HTTPS response object sizes, and single, single and multiple
          transactions per connection scenarios.</t>
          <section numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure the traffic load profile of the test equipment to
            establish the "Initial objective" initial objectives, as defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_Latency" format="default"/>.
            The traffic load profile MUST <bcp14>MUST</bcp14> be defined as described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>The DUT/SUT MUST <bcp14>MUST</bcp14> reach the "Initial objective" initial objectives before the
            sustain phase. The measured KPIs during the sustain phase MUST <bcp14>MUST</bcp14>
            meet all the test results validation criteria defined in <xref target="Validation_Criteria_HTTPS_Latency" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the "Target objective" target objectives
            defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_Latency" format="default"/>.
            The test equipment MUST <bcp14>MUST</bcp14> follow the traffic load profile definition
            as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record all
            specified KPIs. Continue the test until all traffic profile phases
            are completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT MUST <bcp14>MUST</bcp14>
            reach the desired value of the target objective in the sustain
            phase.</t>
            <t>Measure the minimum, average, and maximum values of the TTFB and
            TTLB.</t>
          </section>
        </section>
      </section>
      <section anchor="HTTPS_CC" numbered="true" toc="default">
        <name>Concurrent TCP/QUIC TCP  or QUIC Connection Capacity with HTTPS Traffic</name>
        <section numbered="true" toc="default">
          <name>Objective</name>
          <t>Determine the number of concurrent TCP/QUIC TCP or QUIC connections the
          DUT/SUT sustains when using HTTPS traffic.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Test Setup</name>
          <t>Testbed
          <t>The testbed setup MUST <bcp14>MUST</bcp14> be configured as defined in <xref target="Test_Setup" format="default"/>. Any specific testbed configuration changes
          (number of interfaces and interfaces, interface type, etc.) MUST <bcp14>MUST</bcp14> be
          documented.</t>
        </section>
        <section anchor="HTTPS_CC_parameter" numbered="true" toc="default">
          <name>Test Parameters</name>
          <t>In this section, benchmarking test specific benchmarking-test-specific parameters are
          defined.</t>
          <section numbered="true" toc="default">
            <name>DUT/SUT Configuration Parameters</name>
            <t>DUT/SUT parameters MUST <bcp14>MUST</bcp14> conform to the requirements defined in
            <xref target="DUT-SUT_Configuration" format="default"/>. Any configuration changes
            for this specific benchmarking test MUST <bcp14>MUST</bcp14> be documented.</t>
          </section>
          <section anchor="Test_Equipment_Configuration_Parameters_HTTPS_CC" numbered="true" toc="default">
            <name>Test Equipment Configuration Parameters</name>
            <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
            requirements defined in <xref target="Test_Equipment_Configuration" format="default"/>. The following parameters
            MUST
            <bcp14>MUST</bcp14> be documented for this benchmarking test:</t>
            <ul empty="true" spacing="normal">
              <li>Client IP address ranges defined in <xref target="Client_IP" format="default"/></li>
              <li>Server IP address ranges defined in <xref target="Server_IP" format="default"/></li>
              <li>Traffic distribution ratio between IPv4 and IPv6 defined in
                <xref target="Client_IP" format="default"/></li>
              <li>RECOMMENDED
              <li><bcp14>RECOMMENDED</bcp14> cipher suites and key sizes defined in <xref target="Emulated_web_Browser_attributes" format="default"/></li>
              <li>Target concurrent connections: Initial value from the product
                datasheet or the value defined based on the requirement for a
                specific deployment scenario.</li>
              <li>Initial scenario</li>
              <li><t>Initial concurrent connections: 10% of "Target concurrent
                connections" Note:
                connections"</t>
		<t>Note: Initial concurrent connection connections is not a KPI
                to report. This value is configured on the traffic generator
                and used to perform Step1: "Test Step 1 (Test Initialization and
                Qualification"
                Qualification) described under in <xref target="Test_Procedures_and_Expected_Results_TC_7_9" format="default"/>.</li> format="default"/>.</t></li>
              <li>Connections per second during ramp up phase: 50% of maximum
                connections per second measured in the benchmarking test <xref target="HTTPS_CPS" format="default">TCP/QUIC Connections format="default">TCP or QUIC connections per second with HTTPS
                Traffic</xref></li>
                traffic</xref></li>
              <li>Ramp up time (in traffic load profile for "Target
                concurrent connections"): "Target concurrent connections" /
                "Maximum connections per second during ramp up phase"</li>
              <li>Ramp up time (in traffic load profile for "Initial
                concurrent connections"): "Initial concurrent connections" /
                "Maximum connections per second during ramp up phase"</li>
            </ul> phase"</li></ul>
            <t>The client MUST <bcp14>MUST</bcp14> perform HTTPS transactions with persistence
            persistence, and each client can open multiple concurrent
            connections per server endpoint IP.</t>
            <t>Each client sends 10 GET requests requesting 1 KByte KB HTTPS
            response objects in the same TCP/QUIC TCP or QUIC connections (10
            transactions/connection)
            transactions/connections), and the delay (think time) between each
            transaction MUST <bcp14>MUST</bcp14> be X seconds.</t>
            <t>X seconds, where X is as follows.</t>
	    <t indent="3">X = ("Ramp up time" + "steady state time") /10</t> / 10</t>
            <t>The established connections MUST <bcp14>MUST</bcp14> remain open until the ramp
            down phase of the test. During the ramp down phase, all
            connections MUST <bcp14>MUST</bcp14> be successfully closed with FIN.</t>
          </section>
          <section anchor="HTTPS_CC_Test_Results_Validation_Criteria" numbered="true" toc="default">
            <name>Test Results Validation Criteria</name>
            <t>The following criteria are the test results validation
            criteria. The Test test results validation criteria MUST <bcp14>MUST</bcp14> be monitored
            during the whole sustain phase of the traffic load profile.</t>
            <ol spacing="normal" type="a"><li>Number type="a">
	      <li>The number of failed application transactions (receiving any
                HTTP response code other than 200 OK) MUST <bcp14>MUST</bcp14> be less than 0.001%
                (1 out of 100,000 transactions) of the total attempted
                transactions.</li>
              <li>Number
              <li>The number of terminated TCP connections due to unexpected TCP
                RST
                RSTs sent by the DUT/SUT MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated TCP connections. If HTTP/3 is
                used, the number of terminated QUIC connections due to
                unexpected errors MUST <bcp14>MUST</bcp14> be less than 0.001% (1 out of 100,000
                connections) of the total initiated QUIC connections</li> connections.</li>
              <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
                constant rate (considered (it is considered as a constant rate if any deviation
                of the traffic forwarding rate is less than 5%).</li>
            </ol>
          </section>
          <section anchor="HTTPS_CC_Measurement" numbered="true" toc="default">
            <name>Measurement</name>
            <t>Average Concurrent concurrent TCP or QUIC Connections MUST connections <bcp14>MUST</bcp14> be reported for
            this benchmarking test.</t>
          </section>
        </section>
        <section anchor="Test_Procedures_and_Expected_Results_TC_7_9" numbered="true" toc="default">
          <name>Test Procedures and Expected Results</name>
          <t>The test procedure is designed to measure the concurrent TCP
          connection capacity of the DUT/SUT at the sustaining period of the
          traffic load profile. The test procedure consists of three major
          steps:
          steps. Step 1 ensures the DUT/SUT is able to reach the performance
          value (Initial concurrent connection) and meets the test results
          validation criteria when it was very minimally utilized.  Step 2
          determines whether the DUT/SUT is able to reach the target
          performance value within the test results validation criteria.  Step
          3 determines the maximum achievable performance value within the
          test results validation criteria.</t>
          <t>This test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different
          IPv4 and IPv6 traffic distributions.</t>
          <section anchor="HTTPS_CC_Step1_Test_Initialization" numbered="true" toc="default">
            <name>Step 1: Test Initialization and Qualification</name>
            <t>Verify the link status of all connected physical interfaces.
            All interfaces are expected to be in "UP" status.</t>
            <t>Configure test equipment to establish "Initial concurrent TCP
            connections" defined in <xref target="Test_Equipment_Configuration_Parameters_HTTPS_CC" format="default"/>.
            Except ramp up time, the traffic load profile MUST <bcp14>MUST</bcp14> be defined as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>During the sustain phase, the DUT/SUT MUST <bcp14>MUST</bcp14> reach the "Initial
            concurrent connections". The measured KPIs during the sustain
            phase MUST <bcp14>MUST</bcp14> meet the test results validation criteria "a", "a" and "b"
            defined in <xref target="HTTPS_CC_Test_Results_Validation_Criteria" format="default"/>.</t>
            <t>If the KPI metrics do not meet the test results validation
            criteria, the test procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step
            2".</t> Step
            2.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 2: Test Run with Target Objective</name>
            <t>Configure test equipment to establish the target objective
            ("Target concurrent connections"). The test equipment MUST <bcp14>MUST</bcp14> follow
            the traffic load profile definition (except ramp up time) as
            described in <xref target="Traffic_Load_Profile" format="default"/>.</t>
            <t>During the ramp up and sustain phase, phases, the other KPIs KPIs, such as
            inspected throughput, TCP or QUIC connections per second, and
            application transactions per second MUST NOT second, <bcp14>MUST NOT</bcp14> reach the maximum
            value that the DUT/SUT can support.</t>
            <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record KPIs
            defined in <xref target="HTTPS_CC_Measurement" format="default"/>. Continue the
            test until all traffic profile phases are completed.</t>
            <t>Within the test results validation criteria, the DUT/SUT is
            expected to reach the desired value of the target objective in the
            sustain phase. Follow step 3, Step 3 if the measured value does not meet
            the target value or does not fulfill the test results validation
            criteria.</t>
          </section>
          <section numbered="true" toc="default">
            <name>Step 3: Test Iteration</name>
            <t>Determine the achievable concurrent TCP/QUIC TCP or QUIC connections within
            the test results validation criteria.</t>
          </section>
        </section>
      </section>
    </section>
    <section anchor="IANA" numbered="true" toc="default">
      <name>IANA Considerations</name>
      <t>This document makes no specific request of IANA.</t>
      <t>The IANA
      <t>IANA has assigned IPv4 and IPv6 address blocks in <xref target="RFC6890" format="default"/> that have been registered for special purposes. The
      IPv6 address block 2001:2::/48 has been allocated for the purpose of
      IPv6 Benchmarking benchmarking <xref target="RFC5180" format="default"/> format="default"/>, and the IPv4 address block
      198.18.0.0/15 has been allocated for the purpose of IPv4 Benchmarking benchmarking
      <xref target="RFC2544" format="default"/>. This assignment was made to minimize the
      chance of conflict in case a testing device were to be accidentally
      connected to the part of the Internet.</t>
    </section>
    <section anchor="Security_consieration" numbered="true" toc="default">
      <name>Security Considerations</name>
      <t>The primary goal of this document is to provide benchmarking
      terminology and methodology for next-generation network security devices
      for use in a laboratory isolated laboratory-isolated test environment. However, readers
      should be aware that there is some overlap between performance and
      security issues. Specifically, the optimal configuration for network
      security device performance may not be the most secure, and vice-versa. vice versa.
      Testing security platforms with working exploits and malware carries
      risks. Ensure proper access controls are implemented to prevent
      unintended exposure to vulnerable networks or systems. The cipher suites
      recommended in this document are for test purposes only. The cipher
      suite recommendation for a real deployment is outside the scope of this
      document.</t>
      <t>Security assessment of an NGFW/NGIPS product could also include an
      analysis whether any type of uncommon traffic characteristics would have
      a significant impact on performance. Such performance impacts would
      allow an attacker to use such specifically crafted traffic as a DoS
      attack to reduce the remaining performance available to other traffic
      through the NGFW/NGIPS. Such uncommon traffic characteristics might
      include
      include, for example IP fragmented example, IP-fragmented traffic, a specific type of application
      traffic, or uncommonly high HTTP transaction rate traffic.</t>
    </section>
    <section numbered="true" toc="default">
      <name>Contributors</name>
      <t>The following individuals contributed significantly to the creation
      of this document:</t>
      <t>Alex Samonte, Amritam Putatunda, Aria Eslambolchizadeh, Chao Guo,
      Chris Brown, Cory Ford, David DeSanto, Jurrie Van Den Breekel, Michelle
      Rhines, Mike Jack, Ryan Liles, Samaresh Nair, Stephen Goudreault, Tim
      Carlin, and Tim Otto.</t>
    </section>
    <section anchor="Acknowledgements" numbered="true" toc="default">
      <name>Acknowledgements</name>
      <t>The authors wish to acknowledge the members of NetSecOPEN for their
      participation in the creation of this document. Additionally, the
      following members need to be acknowledged:</t>
      <t>Anand Vijayan, Chris Marshall, Jay Lindenauer, Michael Shannon, Mike
      Deichman, Ryan Riese, and Toulnay Orkun.</t>
    </section>
  </middle>
  <back>
    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>
        <reference anchor="RFC2119" target="https://www.rfc-editor.org/info/rfc2119" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml">
          <front>
            <title>Key words for use in RFCs to Indicate Requirement Levels</title>
            <author fullname="S. Bradner" initials="S." surname="Bradner"/>
            <date month="March" year="1997"/>
            <abstract>
              <t>In many standards track documents several words are used to signify the requirements in the specification.  These words are often capitalized.  This document defines these words as they should be interpreted in IETF documents.  This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.</t>
            </abstract>
          </front>
          <seriesInfo name="BCP" value="14"/>
          <seriesInfo name="RFC" value="2119"/>
          <seriesInfo name="DOI" value="10.17487/RFC2119"/>
        </reference>
        <reference anchor="RFC8174" target="https://www.rfc-editor.org/info/rfc8174" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml">
          <front>
            <title>Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words</title>
            <author fullname="B. Leiba" initials="B." surname="Leiba"/>
            <date month="May" year="2017"/>
            <abstract>
              <t>RFC 2119 specifies common key words that may be used in protocol specifications.  This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.</t>
            </abstract>
          </front>
          <seriesInfo name="BCP" value="14"/>
          <seriesInfo name="RFC" value="8174"/>
          <seriesInfo name="DOI" value="10.17487/RFC8174"/>
        </reference>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
      </references>
      <references>
        <name>Informative References</name>
        <reference anchor="RFC3511" target="https://www.rfc-editor.org/info/rfc3511" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.3511.xml">
          <front>
            <title>Benchmarking Methodology for Firewall Performance</title>
            <author fullname="B. Hickman" initials="B." surname="Hickman"/>
            <author fullname="D. Newman" initials="D." surname="Newman"/>
            <author fullname="S. Tadjudin" initials="S." surname="Tadjudin"/>
            <author fullname="T. Martin" initials="T." surname="Martin"/>
            <date month="April" year="2003"/>
            <abstract>
              <t>This document discusses and defines a number of tests that may be used to describe the performance characteristics of firewalls.  In addition to defining the tests, this document also describes specific formats for reporting the results of the tests.  This document is a product of the Benchmarking Methodology Working Group (BMWG) of the Internet Engineering Task Force (IETF).  This memo provides information for the Internet community.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="3511"/>
          <seriesInfo name="DOI" value="10.17487/RFC3511"/>
        </reference>
        <reference anchor="RFC6815" target="https://www.rfc-editor.org/info/rfc6815" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6815.xml">
          <front>
            <title>Applicability Statement for RFC 2544: Use on Production Networks Considered Harmful</title>
            <author fullname="S. Bradner" initials="S." surname="Bradner"/>
            <author fullname="K. Dubray" initials="K." surname="Dubray"/>
            <author fullname="J. McQuaid" initials="J." surname="McQuaid"/>
            <author fullname="A. Morton" initials="A." surname="Morton"/>
            <date month="November" year="2012"/>
            <abstract>
              <t>The Benchmarking Methodology Working Group (BMWG) has been developing key performance metrics and laboratory test methods since 1990, and continues this work at present.  The methods described in RFC 2544 are intended to generate traffic that overloads network device resources in order to assess their capacity.  Overload of shared resources would likely be harmful to user traffic performance on a production network, and there are further negative consequences identified with production application of the methods.  This memo clarifies the scope of RFC 2544 and other IETF BMWG benchmarking work for isolated test environments only, and it encourages new standards activity for measurement methods applicable outside that scope.  This document is not an Internet Standards Track specification; it is published for informational purposes.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="6815"/>
          <seriesInfo name="DOI" value="10.17487/RFC6815"/>
        </reference>
        <reference anchor="RFC2647" target="https://www.rfc-editor.org/info/rfc2647" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2647.xml">
          <front>
            <title>Benchmarking Terminology for Firewall Performance</title>
            <author fullname="D. Newman" initials="D." surname="Newman"/>
            <date month="August" year="1999"/>
            <abstract>
              <t>This document defines terms used in measuring the performance of firewalls.  It extends the terminology already used for benchmarking routers and switches with definitions specific to firewalls. [STANDARDS-TRACK]</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="2647"/>
          <seriesInfo name="DOI" value="10.17487/RFC2647"/>
        </reference>
        <reference anchor="RFC2544" target="https://www.rfc-editor.org/info/rfc2544" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2544.xml">
          <front>
            <title>Benchmarking Methodology for Network Interconnect Devices</title>
            <author fullname="S. Bradner" initials="S." surname="Bradner"/>
            <author fullname="J. McQuaid" initials="J." surname="McQuaid"/>
            <date month="March" year="1999"/>
            <abstract>
              <t>This document is a republication of RFC 1944 correcting the values for the IP addresses which were assigned to be used as the default addresses for networking test equipment.  This memo provides information for the Internet community.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="2544"/>
          <seriesInfo name="DOI" value="10.17487/RFC2544"/>
        </reference>
        <reference anchor="RFC5180" target="https://www.rfc-editor.org/info/rfc5180" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5180.xml">
          <front>
            <title>IPv6 Benchmarking Methodology for Network Interconnect Devices</title>
            <author fullname="C. Popoviciu" initials="C." surname="Popoviciu"/>
            <author fullname="A. Hamza" initials="A." surname="Hamza"/>
            <author fullname="G. Van de Velde" initials="G." surname="Van de Velde"/>
            <author fullname="D. Dugatkin" initials="D." surname="Dugatkin"/>
            <date month="May" year="2008"/>
            <abstract>
              <t>The benchmarking methodologies defined in RFC 2544 are IP version independent.  However, RFC 2544 does not address some of the specificities of IPv6.  This document provides additional benchmarking guidelines, which in conjunction with RFC 2544, lead to a more complete and realistic evaluation of the IPv6 performance of network interconnect devices.  IPv6 transition mechanisms are outside the scope of this document.  This memo provides information for the Internet community.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="5180"/>
          <seriesInfo name="DOI" value="10.17487/RFC5180"/>
        </reference>
        <reference anchor="RFC6890" target="https://www.rfc-editor.org/info/rfc6890" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6890.xml">
          <front>
            <title>Special-Purpose IP Address Registries</title>
            <author fullname="M. Cotton" initials="M." surname="Cotton"/>
            <author fullname="L. Vegoda" initials="L." surname="Vegoda"/>
            <author fullname="R. Bonica" initials="R." role="editor" surname="Bonica"/>
            <author fullname="B. Haberman" initials="B." surname="Haberman"/>
            <date month="April" year="2013"/>
            <abstract>
              <t>This memo reiterates the assignment of an IPv4 address block (192.0.0.0/24) to IANA.  It also instructs IANA to restructure its IPv4 and IPv6 Special-Purpose Address Registries.  Upon restructuring, the aforementioned registries will record all special-purpose address blocks, maintaining a common set of information regarding each address block.</t>
            </abstract>
          </front>
          <seriesInfo name="BCP" value="153"/>
          <seriesInfo name="RFC" value="6890"/>
          <seriesInfo name="DOI" value="10.17487/RFC6890"/>
        </reference>
        <reference anchor="RFC8200" target="https://www.rfc-editor.org/info/rfc8200" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8200.xml">
          <front>
            <title>Internet Protocol, Version 6 (IPv6) Specification</title>
            <author fullname="S. Deering" initials="S." surname="Deering"/>
            <author fullname="R. Hinden" initials="R." surname="Hinden"/>
            <date month="July" year="2017"/>
            <abstract>
              <t>This document specifies version 6 of the Internet Protocol (IPv6).  It obsoletes RFC 2460.</t>
            </abstract>
          </front>
          <seriesInfo name="STD" value="86"/>
          <seriesInfo name="RFC" value="8200"/>
          <seriesInfo name="DOI" value="10.17487/RFC8200"/>
        </reference>
        <reference anchor="RFC8446" target="https://www.rfc-editor.org/info/rfc8446" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8446.xml">
          <front>
            <title>The Transport Layer Security (TLS) Protocol Version 1.3</title>
            <author fullname="E. Rescorla" initials="E." surname="Rescorla"/>
            <date month="August" year="2018"/>
            <abstract>
              <t>This document specifies version 1.3 of the Transport Layer Security (TLS) protocol. TLS allows client/server applications to communicate over the Internet in a way that is designed to prevent eavesdropping, tampering, and message forgery.</t>
              <t>This document updates RFCs 5705 and 6066, and obsoletes RFCs 5077, 5246, and 6961. This document also specifies new requirements for TLS 1.2 implementations.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="8446"/>
          <seriesInfo name="DOI" value="10.17487/RFC8446"/>
        </reference>
        <reference anchor="RFC9000" target="https://www.rfc-editor.org/info/rfc9000" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9000.xml">
          <front>
            <title>QUIC: A UDP-Based Multiplexed and Secure Transport</title>
            <author fullname="J. Iyengar" initials="J." role="editor" surname="Iyengar"/>
            <author fullname="M. Thomson" initials="M." role="editor" surname="Thomson"/>
            <date month="May" year="2021"/>
            <abstract>
              <t>This document defines the core of the QUIC transport protocol.  QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration.  QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances.  Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="9000"/>
          <seriesInfo name="DOI" value="10.17487/RFC9000"/>
        </reference>
        <reference anchor="RFC9001" target="https://www.rfc-editor.org/info/rfc9001" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9001.xml">
          <front>
            <title>Using TLS to Secure QUIC</title>
            <author fullname="M. Thomson" initials="M." role="editor" surname="Thomson"/>
            <author fullname="S. Turner" initials="S." role="editor" surname="Turner"/>
            <date month="May" year="2021"/>
            <abstract>
              <t>This document describes how Transport Layer Security (TLS) is used to secure QUIC.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="9001"/>
          <seriesInfo name="DOI" value="10.17487/RFC9001"/>
        </reference>
        <reference anchor="RFC9002" target="https://www.rfc-editor.org/info/rfc9002" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9002.xml">
          <front>
            <title>QUIC Loss Detection and Congestion Control</title>
            <author fullname="J. Iyengar" initials="J." role="editor" surname="Iyengar"/>
            <author fullname="I. Swett" initials="I." role="editor" surname="Swett"/>
            <date month="May" year="2021"/>
            <abstract>
              <t>This document describes loss detection and congestion control mechanisms for QUIC.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="9002"/>
          <seriesInfo name="DOI" value="10.17487/RFC9002"/>
        </reference>
        <reference anchor="RFC9113" target="https://www.rfc-editor.org/info/rfc9113" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9113.xml">
          <front>
            <title>HTTP/2</title>
            <author fullname="M. Thomson" initials="M." role="editor" surname="Thomson"/>
            <author fullname="C. Benfield" initials="C." role="editor" surname="Benfield"/>
            <date month="June" year="2022"/>
            <abstract>
              <t>This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced latency by introducing field compression and allowing multiple concurrent exchanges on the same connection.</t>
              <t>This document obsoletes RFCs 7540 and 8740.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="9113"/>
          <seriesInfo name="DOI" value="10.17487/RFC9113"/>
        </reference>
        <reference anchor="RFC9114" target="https://www.rfc-editor.org/info/rfc9114" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9114.xml">
          <front>
            <title>HTTP/3</title>
            <author fullname="M. Bishop" initials="M." role="editor" surname="Bishop"/>
            <date month="June" year="2022"/>
            <abstract>
              <t>The QUIC transport protocol has several features that are desirable in a transport for HTTP, such as stream multiplexing, per-stream flow control, and low-latency connection establishment.  This document describes a mapping of HTTP semantics over QUIC.  This document also identifies HTTP/2 features that are subsumed by QUIC and describes how HTTP/2 extensions can be ported to HTTP/3.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="9114"/>
          <seriesInfo name="DOI" value="10.17487/RFC9114"/>
        </reference>
        <reference anchor="RFC9293" target="https://www.rfc-editor.org/info/rfc9293" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9293.xml">
          <front>
            <title>Transmission Control Protocol (TCP)</title>
            <author fullname="W. Eddy" initials="W." role="editor" surname="Eddy"/>
            <date month="August" year="2022"/>
            <abstract>
              <t>This document specifies the Transmission Control Protocol (TCP).  TCP is an important transport-layer protocol in the Internet protocol stack, and it has continuously evolved over decades of use and growth of the Internet.  Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documented in a piecemeal fashion.  This document collects and brings those changes together with the protocol specification from RFC 793.  This document obsoletes RFC 793, as well as RFCs 879, 2873, 6093, 6429, 6528, and 6691 that updated parts of RFC 793.  It updates RFCs 1011 and 1122, and it should be considered as a replacement for the portions of those documents dealing with TCP requirements.  It also updates RFC 5961 by adding a small clarification in reset handling while in the SYN-RECEIVED state.  The TCP header control bits from RFC 793 have also been updated based on RFC 3168.</t>
            </abstract>
          </front>
          <seriesInfo name="STD" value="7"/>
          <seriesInfo name="RFC" value="9293"/>
          <seriesInfo name="DOI" value="10.17487/RFC9293"/>
        </reference>
        <reference anchor="RFC9204" target="https://www.rfc-editor.org/info/rfc9204" xml:base="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9204.xml">
          <front>
            <title>QPACK: Field Compression for HTTP/3</title>
            <author fullname="C. Krasic" initials="C." surname="Krasic"/>
            <author fullname="M. Bishop" initials="M." surname="Bishop"/>
            <author fullname="A. Frindell" initials="A." role="editor" surname="Frindell"/>
            <date month="June" year="2022"/>
            <abstract>
              <t>This specification defines QPACK: a compression format for efficiently representing HTTP fields that is to be used in HTTP/3.  This is a variation of HPACK compression that seeks to reduce head-of-line blocking.</t>
            </abstract>
          </front>
          <seriesInfo name="RFC" value="9204"/>
          <seriesInfo name="DOI" value="10.17487/RFC9204"/>
        </reference>

<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.3511.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6815.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2647.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2544.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.5180.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6890.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8200.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8446.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9000.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9001.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9002.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9113.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9114.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9293.xml"/>
<xi:include href="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9204.xml"/>

        <reference anchor="Wiki-NGFW" target="https://en.wikipedia.org/wiki/Next-generation_firewall"> target="https://en.wikipedia.org/w/index.php?title=Next-generation_firewall&amp;oldid=1133673904">
          <front>
            <title/>
            <author/>
            <date/>
            <title>Next-generation firewall</title>
            <author>
	    <organization>Wikipedia</organization>
	    </author>
            <date month="January" year="2023"/>
          </front>
        </reference>

        <reference anchor="fastly" target="https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency">
          <front>
            <title>Can QUIC match TCP's computational efficiency?</title>
            <title>QUIC vs TCP: Which is Better?</title>
            <author fullname="Kazuho Oku"/>
            <author fullname="Jana Iyengar"/>
            <date day="30" month="April" year="2020"/>
          </front>
        </reference>

        <reference anchor="Undertow" target="https://undertow.io/blog/2015/04/27/An-in-depth-overview-of-HTTP2.html">
          <front>
            <title>An in depth overview of HTTP/2</title>
            <author/>
            <date/>
            <author>
	      <organization>undertow</organization>
	    </author>
          </front>
        </reference>

        <reference anchor="CVE" target="https://www.cvedetails.com/">
          <front>
            <title>Current CVSS Score Distribution For All Vulnerabilities</title>
            <author>
              <organization>CVE</organization>
            </author>
          </front>
        </reference>

      </references>
    </references>

    <section anchor="Test-Methodology-Security-Effectiveness-Evaluation" numbered="true" toc="default">
      <name>Test Methodology - Security Effectiveness Evaluation</name>
      <section numbered="true" toc="default">
        <name>Test Objective</name>
        <t>This test methodology verifies the DUT/SUT is able to detect,
        prevent, and report the vulnerabilities.</t>
        <t>In this test, background test traffic will be generated to utilize
        the DUT/SUT. In parallel, a number of some malicious traffic will be sent
        to the DUT/SUT as encrypted and as well as clear text cleartext payload formats
        using a traffic generator. <xref target="security_effectiveness" format="default"/>
        defines the selection of the malicious traffic from the Common
        Vulnerabilities and Exposures (CVE) (CVEs) list for testing.</t>
        <t>The following KPIs are measured in this test:</t>
        <ul spacing="normal">
          <li>Number of blocked CVEs</li>
          <li>Number of bypassed (nonblocked) (non-blocked) CVEs</li>
          <li>Background traffic performance (verify if the background
            traffic is impacted while sending CVE CVEs toward the DUT/SUT)</li>
          <li>Accuracy of DUT/SUT statistics in terms of vulnerabilities
            reporting</li>
        </ul>
      </section>
      <section numbered="true" toc="default">
        <name>Testbed Setup</name>
        <t>The same testbed MUST <bcp14>MUST</bcp14> be used for security effectiveness tests and
        as well as
        for benchmarking test cases defined in <xref target="Benchmarking" format="default"/>.</t>
      </section>
      <section numbered="true" toc="default">
        <name>Test Parameters</name>
        <t>In this section, the benchmarking test specific benchmarking-test-specific parameters are
        defined.</t>
        <section numbered="true" toc="default">
          <name>DUT/SUT Configuration Parameters</name>
          <t>DUT/SUT configuration parameters MUST <bcp14>MUST</bcp14> conform to the requirements
          defined in <xref target="DUT-SUT_Configuration" format="default"/>. The same DUT
          configuration MUST <bcp14>MUST</bcp14> be used for the security effectiveness test and
          as well as
          for benchmarking test cases defined in <xref target="Benchmarking" format="default"/>. The DUT/SUT MUST <bcp14>MUST</bcp14> be configured in inline
          mode and "Inline"
          mode, all detected attack traffic MUST <bcp14>MUST</bcp14> be dropped dropped, and the session
          MUST
          <bcp14>MUST</bcp14> be reset</t>
        </section>
        <section anchor="Test_Equipment_Configuration_Parameters_CVE" numbered="true" toc="default">
          <name>Test Equipment Configuration Parameters</name>
          <t>Test equipment configuration parameters MUST <bcp14>MUST</bcp14> conform to the
          requirements defined in <xref format="default" target="Test_Equipment_Configuration"/>. The same client and server
          IP ranges MUST <bcp14>MUST</bcp14> be configured as used in the benchmarking test cases.
          In addition, the following parameters MUST <bcp14>MUST</bcp14> be documented for this
          benchmarking test:</t>
          <ul spacing="normal">
            <li>Background Traffic: 45% of maximum HTTP throughput and 45% of
              Maximum
              maximum HTTPS throughput supported by the DUT/SUT (measured with
              object size 64 KByte KB in the benchmarking tests "HTTP(S)
              Throughput" HTTP(S)
              Throughput defined in Sections <xref target="HTTP_TP" format="default"/> format="counter"/>
              and <xref target="HTTPS_TP" format="default"/>).</li>
            <li>RECOMMENDED format="counter"/>)</li>
            <li><bcp14>RECOMMENDED</bcp14> CVE traffic transmission Rate: 10 CVEs per
              second</li>
            <li>It is RECOMMENDED <bcp14>RECOMMENDED</bcp14> to generate each CVE multiple times
              (sequentially) at 10 CVEs per second</li> second.</li>
            <li>Ciphers and keys for the encrypted CVE traffic MUST <bcp14>MUST</bcp14> use the
              same cipher configured for HTTPS traffic related HTTPS-traffic-related benchmarking
              tests (<xref target="HTTPS_CPS" format="default"/> - (Sections <xref target="HTTPS_CPS" format="counter"/>-<xref target="HTTPS_CC" format="default"/>)</li> format="counter"/>)</li>
          </ul>
        </section>
      </section>
      <section anchor="CVE_Criteria" numbered="true" toc="default">
        <name>Test Results Validation Criteria</name>
        <t>The following criteria are the test results validation criteria.
        The test results validation criteria MUST <bcp14>MUST</bcp14> be monitored during the
        whole test duration.</t>
        <ol spacing="normal" type="a"><li>Number type="a">
	  <li>The number of failed application transactions in the background
            traffic MUST <bcp14>MUST</bcp14> be less than 0.01% of the attempted transactions.</li>
          <li>Number
          <li>The number of terminated TCP or QUIC connections of the background
            traffic (due to unexpected errors) MUST <bcp14>MUST</bcp14> be less than 0.01% of the
            total initiated TCP connections in the background traffic.</li>
          <li>During the sustain phase, traffic MUST <bcp14>MUST</bcp14> be forwarded at a
            constant rate (considered (it is considered as a constant rate if any deviation of the
            traffic forwarding rate is less than 5%).</li>
          <li>False
          <li>A false positive MUST NOT <bcp14>MUST NOT</bcp14> occur in the background traffic.</li>
        </ol>
      </section>
      <section numbered="true" toc="default">
        <name>Measurement</name>
        <t>The following KPI metrics MUST <bcp14>MUST</bcp14> be reported for this test
        scenario:</t>
        <t>Mandatory KPIs:</t>
        <ul spacing="normal">
          <li>
            <t>Blocked CVEs: They MUST <bcp14>MUST</bcp14> be represented in the
            following ways:</t>
            <ul spacing="normal">
              <li>Number of blocked CVEs out of total CVEs</li>
              <li>Percentage of blocked CVEs</li>
            </ul>
          </li>
          <li>
            <t>Unblocked CVEs: They MUST <bcp14>MUST</bcp14> be represented in the
            following ways:</t>
            <ul spacing="normal">
              <li>Number of unblocked CVEs out of total CVEs</li>
              <li>Percentage of unblocked CVEs</li>
            </ul>
          </li>
          <li>
            <t>Background traffic behavior: It MUST <bcp14>MUST</bcp14> be
            represented in one of the followings ways:</t>
            <ul spacing="normal">
              <li>No impact: Considered as "no impact'" impact" if any deviation of the
              traffic forwarding rate is less than or equal to 5 % 5% (constant
              rate)</li>
              <li>Minor impact: Considered as "minor impact" if any deviation
              of the traffic forwarding rate is greater than 5% and less than or
              equal to10% (i.e. to 10% (i.e., small spikes)</li>
              <li>Heavily impacted:
              <li>Heavy impact: Considered as "Heavily impacted" "heavy impact" if any
              deviation of the traffic forwarding rate is greater than 10% (i.e. (i.e.,
              large spikes) or reduced the background HTTP(S) throughput
              greater than 10%</li>
            </ul>
          </li>
          <li>DUT/SUT reporting accuracy: The DUT/SUT MUST <bcp14>MUST</bcp14> report
          all detected vulnerabilities.</li>
        </ul>
        <t>Optional KPIs:</t>
        <ul spacing="normal">
          <li>List of unblocked CVEs</li>
        </ul>

      </section>
      <section numbered="true" toc="default">
        <name>Test Procedures and Expected Results</name>
        <t>The test procedure is designed to measure the security
        effectiveness of the DUT/SUT at the sustaining period of the traffic
        load profile. The test procedure consists of two major steps. This
        test procedure MAY <bcp14>MAY</bcp14> be repeated multiple times with different IPv4 and
        IPv6 traffic distributions.</t>
        <section numbered="true" toc="default">
          <name>Step 1: Background Traffic</name>
          <t>Generate background traffic at the transmission rate defined in
          <xref format="default" target="Test_Equipment_Configuration_Parameters_CVE"/>.</t>
          <t>The DUT/SUT MUST <bcp14>MUST</bcp14> reach the target objective (HTTP(S) throughput)
          in the sustain phase. The measured KPIs during the sustain phase MUST <bcp14>MUST</bcp14>
          meet all the test results validation criteria defined in <xref target="CVE_Criteria" format="default"/>.</t>
          <t>If the KPI metrics do not meet the acceptance test results validation criteria, the test
          procedure MUST NOT <bcp14>MUST NOT</bcp14> be continued to "Step 2".</t> Step 2.</t>
        </section>
        <section numbered="true" toc="default">
          <name>Step 2: CVE Emulation</name>
          <t>While generating background traffic (in the sustain phase), send the
          CVE traffic traffic, as defined in the parameter section.</t> section (<xref target="Test_Equipment_Configuration_Parameters_CVE"/>).</t>
          <t>The test equipment MUST <bcp14>MUST</bcp14> start to measure and record all specified
          KPIs. Continue the test until all CVEs are sent.</t>
          <t>The measured KPIs MUST <bcp14>MUST</bcp14> meet all the test results validation
          criteria defined in <xref target="CVE_Criteria" format="default"/>.</t>
          <t>In addition, the DUT/SUT should either report the detected
   vulnerabilities in the log correctly correctly, or if, for example, a different
          naming convention is used, there MUST <bcp14>MUST</bcp14> be reference
   material available that will allow for verification that the correct
   vulnerability was detected. detected if, for example, a different naming
   convention is used. This reference material MUST <bcp14>MUST</bcp14> be cited in
          the report.</t>
        </section>
      </section>
    </section>
    <section anchor="DUT-Classification" numbered="true" toc="default">
      <name>DUT/SUT Classification</name>
      <t>This document aims to classify the DUT/SUT into four different
      categories based on its maximum supported maximum-supported firewall throughput
      performance number defined in the vendor datasheet. This classification
      MAY
      <bcp14>MAY</bcp14> help users to determine specific configuration scales (e.g., number
      of ACL entries), traffic profiles, and attack traffic profiles, scaling
      those proportionally to the DUT/SUT sizing category.</t>
      <t>The four different categories are Extra Small (XS), Small (S), Medium
      (M), and Large (L). The RECOMMENDED <bcp14>RECOMMENDED</bcp14> throughput values for the following
      categories are:</t>
      <t>Extra
      <dl newline="false" spacing="normal">
	<dt>Extra Small (XS) - Supported -</dt>
	<dd>Supported throughput less than or equal
      to1Gbit/s</t>
      <t>Small to 1 Gbit/s</dd>
	<dt>Small (S) - Supported -</dt>
	<dd>Supported throughput greater than 1Gbit/s 1 Gbit/s and less than or equal
	to 5Gbit/s</t>
      <t>Medium 5Gbit/s</dd>
	<dt>Medium (M) - Supported -</dt>
	<dd>Supported throughput greater than 5Gbit/s 5 Gbit/s and less than or equal
	to 10Gbit/s</t>
      <t>Large 10Gbit/s</dd>
	<dt>Large (L) - Supported -</dt>
	<dd>Supported throughput greater than 10Gbit/s</t> 10 Gbit/s</dd>
      </dl>
    </section>
    <section anchor="Acknowledgements" numbered="false" toc="default">
      <name>Acknowledgements</name>
      <t>The authors wish to acknowledge the members of NetSecOPEN for their
      participation in the creation of this document. Additionally, the
      following members need to be acknowledged:</t>
      <t><contact fullname="Anand Vijayan"/>, <contact fullname="Chris
      Marshall"/>, <contact fullname="Jay Lindenauer"/>, <contact
      fullname="Michael Shannon"/>, <contact fullname="Mike Deichman"/>,
      <contact fullname="Ryan Riese"/>, and <contact fullname="Toulnay
      Orkun"/>.</t>
    </section>
    <section numbered="false" toc="default">
      <name>Contributors</name>
      <t>The following individuals contributed significantly to the creation
      of this document:</t>
      <t><contact fullname="Alex Samonte"/>, <contact fullname="Amritam
      Putatunda"/>, <contact fullname="Aria Eslambolchizadeh"/>, <contact
      fullname="Chao Guo"/>, <contact fullname="Chris Brown"/>, <contact
      fullname="Cory Ford"/>, <contact fullname="David DeSanto"/>, <contact
      fullname="Jurrie Van Den Breekel"/>, <contact fullname="Michelle
      Rhines"/>, <contact fullname="Mike Jack"/>, <contact fullname="Ryan
      Liles"/>, <contact fullname="Samaresh Nair"/>, <contact
      fullname="Stephen Goudreault"/>, <contact fullname="Tim Carlin"/>, and
      <contact fullname="Tim Otto"/>.</t>
    </section>
  </back>
</rfc>