Skip to content

Benchmark methodologies we run on the bench

Five standards with a page each: RFC 2544 for throughput and latency, RFC 2889 for switches, RFC 9411 with RFC 3511 for firewalls, RFC 6349 for TCP over a real path and ITU-T Y.1564 for Ethernet services. Each page gives the test bed, the parameter table with ranges and recommended values, the procedure, the report structure and 5–6 questions buyers ask. Three more standards run on request.

Request a test plan See CloudLAB
Which standard for which question pick the row that sounds like your RFP
How much does the box forward? throughput, latency, loss, back-to-back RFC 2544 How does the switch cope with congestion? MAC learning, head-of-line blocking, broadcast RFC 2889 How many sessions can the firewall hold? CPS, concurrent sessions, L7 throughput RFC 9411 / 3511 Why is TCP slow on a 1 Gbit link? BDP, window, efficiency, buffer delay RFC 6349 Does the Ethernet service meet its SLA? CIR, EIR, delay, jitter, loss ITU-T Y.1564
  • 5standards with a page
  • 3more on request
  • 1–5 daysbench time per unit
  • 100GL2-3 ports, 200G+ for L4-7

Index

What each page covers

Same structure on every page, so two methodologies can be compared side by side and the right one dropped into an RFP without rewriting

StandardMetricsDevice under testBench setupTime
01RFC 2544 benchmarking Throughput, latency, frame loss rate, back-to-back frames at 64–9000 bytes Router, L3 switch, firewall in forwarding mode Generator port pair at 1G, 10G or 100G, bidirectional 1–3 days
02RFC 2889 switch benchmarking Forwarding rate, address learning and caching, congestion control, broadcast forwarding LAN, campus and data-centre switch Full mesh or partial mesh over 4–32 ports 2–3 days
03RFC 9411 and RFC 3511 firewall benchmarking TCP/HTTP connections per second, concurrent connections, L7 throughput, TLS handshake rate Firewall, NGFW, IPS, with policy and inspection on L4-7 engine on both sides of the unit, 10G to 200G+ 3–5 days
04RFC 6349 TCP throughput Path MTU, baseline RTT, bottleneck bandwidth, TCP throughput, efficiency, buffer delay WAN link, SD-WAN path, managed service, cloud connection Two endpoints over the real path or a WAN emulator 1–2 days
05ITU-T Y.1564 service activation Information rate, frame transfer delay, delay variation, frame loss, availability per service Carrier Ethernet service, EVC, business access Generator and loopback or second unit at the far end 2–24 h of traffic

On request

Three more we run without a page yet

The plan is written the same way; the page follows when enough projects have used it

  1. RFC 5180

    IPv6 benchmarking: the RFC 2544 set with IPv6 headers, extension headers and dual-stack mixes

  2. ITU-T Y.1731

    Ethernet OAM: delay, jitter and loss measurement frames for SLA monitoring between maintenance points

  3. TR-398

    Wi-Fi in-premises performance: throughput, coverage, multi-client and roaming on the Wi-Fi bench up to Wi-Fi 6

How a page is built

One structure, read in five minutes

A buyer reads the first paragraph and the table; an engineer reads the procedure and the report section; both see the same numbers

  1. Who it is for

    The purchase or rollout decision the standard supports, and the device class it fits

  2. Test bed and parameters

    Scheme of the setup, a table with each parameter, its range and the value we recommend, and why

  3. Procedure and report

    Step list with durations, what the report contains, and how pass or fail is decided against your thresholds

Questions

Choosing a methodology

RFC 2544 RFC 2889 RFC 9411 / 3511 RFC 6349 Y.1564

RFC 2544 or Y.1564 for a carrier Ethernet service?

Y.1564 when the question is whether a service with a CIR and an EIR meets its SLA, several services at once, within hours. RFC 2544 when the question is the raw limit of one device, one frame size at a time. Many activation projects use Y.1564 for the service and RFC 2544 for the box behind it.

RFC 3511 or RFC 9411 for a firewall?

RFC 9411 is the current methodology and replaces RFC 3511: HTTPS traffic, realistic object sizes, inspection on, and separate numbers for connections per second, concurrent connections and throughput. We quote both names because tenders still cite RFC 3511; the test follows RFC 9411.

Why does a 1 Gbit link give 300 Mbit/s in a file transfer?

Usually the TCP window is smaller than the bandwidth-delay product, or a buffer along the path adds delay under load. RFC 6349 measures exactly that: the MTU, the RTT, the achievable TCP throughput and how far it sits from the line rate, with an efficiency and a buffer-delay figure that point at the cause.

Can two methodologies run in one project?

Yes, and it is the usual case. A switch bake-off combines RFC 2544 for the port pair and RFC 2889 for the mesh; a firewall project adds RFC 2544 in forwarding mode to RFC 9411 with inspection on. One plan, one report.

Which frame sizes and durations do you use?

The sizes the standard lists, 64 to 1518 bytes plus jumbo up to 9000 where the device supports it, and trial durations of 60 seconds by default with longer soak runs where the plan asks for them. The parameter table on each page shows the ranges and the values we recommend.

Pick the standard, we write the plan

Name the device and the standard, or just the question. A test plan draft comes within one business day, a quote in 1–3 business days.

Request a test plan See CloudLAB

Send the shortlist

You have the method. Now get the numbers.

Name the models and the KPIs that matter. An engineer replies within one business day with a test plan draft and a quote.

Prefer to chat? Message an engineer on WhatsApp

We use what you send only to answer you. See our Privacy Policy.