Skip to content

Methodology · Firewall and NGFW benchmarking

RFC 9411: the firewall test with every feature switched on

RFC 9411 replaced RFC 3511 in 2023. It benchmarks a firewall the way you run one: inline, TLS inspection and IPS enabled, real HTTP and HTTPS traffic, and a failure budget of 1 transaction in 100,000. This page walks through the method as we run it in CloudLAB and what the report gives you.

  • 9 benchmark tests: 5 over HTTP, 4 over HTTPS with the recommended cipher suites
  • 300 s sustain phase per run, KPIs sampled every 2 s or faster
  • A result counts only if failed transactions and unexpected RSTs stay under 0.001%
Get an NGFW test plan Jump to the 9 tests
OFFERED LOAD TRAFFIC LOAD PROFILE · RFC 9411 §4.3.4 TARGET OBJECTIVE 10% · QUALIFICATION KPIs MEASURED HERE SAMPLE INTERVAL UNDER 2 s INIT ≥ 5 s RAMP UP SUSTAIN ≥ 300 s RAMP DOWN COLLECT Only the sustain phase makes it into the report

Every RFC 9411 run follows this profile. Measurements happen in the sustain phase only: 300 s minimum, every KPI recorded there and nowhere else.

Which RFC

RFC 9411 is the method. RFC 3511 is history.

RFC 3511 dates from 2003 and measured a firewall as a packet filter: IP throughput, connection rates, a rule set. RFC 9411 obsoleted it in 2023. Everything on this page is RFC 9411; 3511 comes up only when a tender still quotes it.

Inspected throughput

The KPI is bits that were examined, allowed and delivered while the DUT inspects. Layer 2 by default; if you measure at layer 3, the report must say so, because the two are not comparable.

One configuration for all 9 tests

The DUT configuration is frozen before the first test and stays identical to the last one. Every recommended feature that is off goes into the report together with the reason and the performance implication.

Worst case on purpose

Full TLS handshake on every connection, session resumption off, PSH set on all traffic. The method looks for the floor of the device, the number your users will actually get on a bad day.

Free template

RFC 9411 test plan template

The plan we fill in before every firewall and NGFW run, as a PDF: device class and access list, the security features to switch on, the 9 tests with object sizes and cipher suites, the validation gate and a results sheet.

  • Ready to attach to a tender or an RFP
  • The same parameters and pass criteria for every vendor and every lab
  • The mistakes that change the result, listed at the end

PDF in English. We ask only for your email.

Test setup

Two halves of a traffic generator and one box in between

Option 1 in the RFC: emulated clients on one side, emulated servers on the other, the DUT inline. No external switches or routers unless the port count forces them: they add their own latency and shrink the ARP table the DUT has to cope with.

TEST EQUIPMENT · CLIENTS EMULATED ROUTER (OPTIONAL) EMULATED BROWSERS HTTP/1.1 · HTTP/2 · HTTP/3 over QUIC TLS 1.2 / 1.3, full handshake, no resumption MSS 1460 · window 65535 · cwnd ≤ 10 MSS Source ports 1024–65535, PSH on 1,400–1,700 CLIENT IPs PER 10 Gbit/s GET · TLS DUT / SUT INLINE · FAIL-OPEN OFF · FLOW LOGGING ON One configuration for all 9 tests ACL set sized by class XS / S / M / L NGFW · NGIPS · PHYSICAL OR VNF TLS INSPECTION IDS/IPS ANTI-VIRUS ANTI-SPYWARE GEO ANTI-BOTNET ANTI-EVASION APP-ID LOGGING 200 OK TEST EQUIPMENT · SERVERS EMULATED ROUTER (OPTIONAL) EMULATED SERVER POOL HTTP on :80, HTTPS on :443, HTTP/3 on UDP TLS 1.2 or newer, 16 KB records No session tickets, no session cache One IP per FQDN per test port IPv4 / IPv6 SPLIT MIRRORS THE CLIENTS REFERENCE PRE-TEST FIRST same bed, DUT replaced by a patch cable or set to fast-forward · loss, latency and generator headroom recorded before the DUT goes in

Emulated clients

MSS 1460 (IPv4) and 1440 (IPv6), 65,535-byte windows, initial congestion window up to 10 MSS, 3 retries, source ports 1024 to 65535. The client count follows the target: about 1,400 to 1,700 IPs per 10 Gbit/s for an enterprise perimeter, 50,000 to 100,000 for a subscriber profile.

DUT / SUT

Inline, fail-open off, recommended features on, flow-level logging on, geo filtering where the box supports it. The ACL set is sized by class: from 5 application rules and 50 IP and port blocks on an XS device to 50 and 500 on a Large one. Block rules first, allow rules last, none of them touching the test traffic.

Emulated servers

HTTP/1.1 and HTTP/2 on ports 80 and 443, HTTP/3 on UDP. TLS 1.2 or newer, 16 KB record limit, no session tickets and no session cache, one IP per server FQDN per test port. The IPv4 to IPv6 split mirrors the client side.

Before the DUT goes in: the reference pre-test

We run the smallest-object HTTP profile through a patch cable (or the DUT in fast-forward mode), record the loss and latency of the bed itself and confirm the generator clears the DUT's claimed performance with headroom. If the bed cannot beat the datasheet, the DUT result is meaningless. Virtual beds get an extra watch for noisy neighbours, VM moves and thermal throttling during the 300 s.

DUT/SUT configuration

What has to be on before the first packet

Section 4.2 of the RFC. The list below is the NGFW one; an NGIPS gets a shorter list (no anti-virus, no ACLs) with DPI and anti-malware added. Machine-learning and behavioural engines are switched off: the method cannot reproduce them.

Recommended: on for every test

  • TLS Inspection
  • IDS / IPS
  • Anti-Virus
  • Anti-Spyware
  • Anti-Botnet
  • Anti-Evasion
  • Application Identification

Switching one of these off is allowed when the deployment does not need it. The report then states which, why, and that the result would be lower with it on. Application identification must be configured to recognise the applications in the traffic mix.

Optional: by deployment

  • Web Filtering
  • Data Loss Protection
  • DDoS Protection
  • Certificate Validation

On if your policy runs them in production. We usually test both ways when one of them is a buying criterion, DDoS protection in particular.

Access list by device class (Table 4 of the RFC)

Class comes from the throughput on the vendor datasheet. The rules go in this order, specific denies on top, and none of them may match the test traffic.

Rule XS
≤ 1 Gbit/s		 S
1–5 Gbit/s		 M
5–10 Gbit/s		 L
> 10 Gbit/s
Block by application (applications absent from the traffic mix)5102050
Block by source IP and destination port (ports not used in the test)2550100250
Block by source and destination subnet (subnets not used in the test)2550100250
Allow by application (half of the applications in the mix)10101010
Allow by client subnet and port, then by client subnetat least 1 rule per client subnet, in both rule types

Classes: XS up to 1 Gbit/s, S 1 to 5 Gbit/s, M 5 to 10 Gbit/s, L above 10 Gbit/s (Appendix B). Your own rule count can be higher; the RFC sets the floor. If the box is built without ACLs (most NGIPS), we test without them and say so in the report.

Section 7 of the RFC

9 tests, each with its own target and its own gate

5 run over cleartext HTTP, 4 repeat the pattern over HTTPS with the recommended cipher suites. Every test starts at 10% of the target, then runs at the target, then iterates to the highest load that still passes the validation criteria.

  1. 7.1Application mix

    Throughput with application traffic mix

    Inspected throughput and transactions per second under a realistic mix, for example the NetSecOPEN enterprise perimeter profile or a capture of your own perimeter. The closest thing to a production day in a single number.

    Target: line rate of the DUT ports or your deployment figure · TTLB reported with the object sizes of the mix
  2. 7.2HTTP

    TCP connections per second

    One GET per connection, FIN straight after. Shows how fast the session table, the policy lookup and the logging pipeline can take new flows. Small objects stress the control path, 64 KB starts pulling throughput into the picture.

    Objects: 1, 2, 4, 16 and 64 KB, one size per iteration · target: datasheet CPS or your own
  3. 7.3HTTP

    HTTP throughput

    Sustainable inspected throughput while the object size varies. Small objects show what the inspection engines cost per request; 256 KB shows what the data path can push when the engines have little to do.

    Objects: 1, 16, 64, 256 KB plus a mixed set from 0.2 KB up · throughput and transactions/s per size
  4. 7.4HTTP

    HTTP transaction latency

    Time to first byte and time to last byte while the box runs at half of what tests 7.2 and 7.3 found. Latency at 50% load is the figure your users feel; latency at idle is the one on the datasheet.

    TTFB and TTLB min / avg / max · 1 and 10 transactions per connection · objects 1, 16 and 64 KB
  5. 7.5HTTP

    Concurrent TCP connection capacity

    How many sessions the DUT keeps open at once without dropping or resetting any of them. The ramp-up is paced by the CPS result from 7.2 so the count itself is what breaks, not the setup rate.

    Ramp-up time = target connections ÷ max CPS · sustain checks the count stays within 10%
  6. 7.6HTTPS · TLS inspected

    TCP or QUIC connections per second, HTTPS

    Test 7.2 with a full TLS handshake on every connection and the DUT decrypting in the middle. Run per cipher suite and key strength; RSA 4096 with P-384 is where proxy-mode inspection hurts most.

    Objects: 1, 2, 4, 16 and 64 KB · TLS handshake rate measured on the 1 KB object
  7. 7.7HTTPS · TLS inspected

    HTTPS throughput

    Test 7.3 over TLS. This is the number to put next to the datasheet: decrypt, inspect, re-encrypt at 64 KB objects with IPS and application identification on.

    Iterations cover ECDHE with AES-128-GCM and AES-256-GCM, RSA 2048 and 4096, P-256 and P-384
  8. 7.8HTTPS · TLS inspected

    HTTPS transaction latency

    TTFB and TTLB with the TLS proxy doing its work at 50% load. Expect the TTFB to move the most: the handshake now happens twice, once to the DUT and once from it.

    Same scenarios as 7.4 · min / avg / max per object size and cipher suite
  9. 7.9HTTPS · TLS inspected

    Concurrent TCP or QUIC connection capacity, HTTPS

    Open TLS sessions held at once. With TLS 1.3 the DUT works as a middlebox that holds the certificates and keys, so the session table and the key material both count against the limit.

    Ramp-up paced by the HTTPS CPS from 7.6 · deviation under 10% during sustain

Procedure

Every test is a search for the ceiling, with a gate at each step

The RFC does not let you report a number that broke something along the way. 4 moves, in this order, for each of the 9 tests and each object size.

  1. 00

    Reference pre-test

    Bed without the DUT. Loss, latency and generator headroom recorded. Repeated if anything in the bed changes during the project.

  2. 01

    Qualification at 10%

    Initial objective is 10% of the target. All KPIs must pass the criteria at this light load, or the test stops here and we go debugging the configuration.

  3. 02

    Run at target

    Full load profile at the target objective: datasheet value, line rate or your own requirement. Passed within the criteria? That is the result.

  4. 03

    Iterate to the maximum

    Missed the target or broke a criterion? Step down and repeat until the highest load that passes is found. The final iteration runs the full 300 s sustain.

The validation gate, monitored through the whole sustain phase

  • 0.001%

    Maximum share of failed transactions: anything other than 200 OK, counted against all attempts. 1 in 100,000.

  • 0.001%

    Maximum share of connections torn down by an unexpected RST from the DUT. A firewall that resets under load fails the run.

  • 5%

    Maximum deviation of the forwarding rate during sustain. The traffic has to flow flat, not in sawtooth bursts.

  • 10%

    Maximum deviation of concurrent connections in steady state. Confirms the DUT opens and closes sessions at the same pace.

The load profile behind every run

  • Init, 5 s or moreARP/ND and MAC learning. Clients stay silent, servers are ready. No measurement.
  • Ramp upFrom 0 to the target objective, slow enough that the generator never exceeds the DUT's own CPS or throughput limits. No measurement.
  • Sustain, 300 s or moreConstant load, constant client count. The only phase that is measured: every KPI recorded continuously, sampling interval under 2 s.
  • Ramp down and collectionTraffic back to 0, then the generator merges counters and statistics into the report data. No measurement.

Deliverable

What lands on your desk

Section 6 of the RFC defines the report. Ours follows it and adds the raw material, so an auditor, a vendor engineer or your own team can re-check every number without calling us.

Benchmarks per test

  • Inspected throughput in Gbit/s to 2 decimals, with kbit/s in brackets
  • TCP or QUIC connections per second, per object size
  • Application transactions per second (200 OK only)
  • TLS handshake rate per cipher suite and key strength
  • Concurrent TCP or QUIC connections held
  • TTFB and TTLB: minimum, average, maximum, per object size

Context that makes the numbers comparable

  • DUT make, model, firmware, port speeds; for a VNF: cores, RAM, DPDK or SR-IOV, pinning, NUMA, hypervisor
  • Every enabled feature, and the reason for every recommended feature that is off
  • Generator model, software version, TCP and TLS parameters, cipher suites
  • Traffic mix, IPv4 to IPv6 ratio, client and server address ranges, sweep method
  • Reference pre-test results: loss and latency of the bed itself
  • Time and date of every run, phase by phase
RFC 9411 · TEST REPORT SUSTAIN 300 s · IPv4 100% 7.1Throughput, app mix 7.2TCP CPS, 1 KB object 7.4TTLB, 64 KB object 7.6HTTPS CPS, ECDHE-RSA 2048 7.7HTTPS throughput, 64 KB 7.9HTTPS concurrent capacity 6.20 Gbit/s 48,000 /s 41 ms avg 9,800 /s 5.40 Gbit/s 1.8 M transactions 5,900 /s · TTLB with mix object sizes failed 0.0003% · RST 0.0001% · rate dev 1.2% TTFB 9 / 12 / 31 ms · 10 transactions per connection TLS handshakes 9,800 /s · full handshake, no resumption AES-128-GCM · P-256 · inspection on RST 0.004% at 2.0 M: criterion b failed, 1.8 M reported ATTACHED DUT CONFIG EXPORT FEATURES + REASONS PRE-TEST RESULTS RAW COUNTERS · PCAP Values are illustrative. Your report carries your numbers, per object size and cipher suite.

From us, on top of the RFC

  • Raw counters and pcaps from every iteration, so the numbers can be re-derived
  • The DUT configuration export as tested, with the ACL set
  • A 1-page recommendation: which box, at what load, with what switched on

From practice

40 Gbit/s on the datasheet. Which method produced it?

The headline number of most NGFWs comes from an RFC 3511-style run: large IP packets, a handful of sessions, inspection off. Put the same box through RFC 9411 with TLS inspection, IPS and application identification on, 64 KB HTTPS objects and a full handshake per connection, and the inspected throughput lands at 6 Gbit/s. The box measured 40 with one ruler and 6 with the other.

Buy for the 40 and the perimeter is sized 6 to 7 times too small. We have seen what happens next: the first outage, and an engineer quietly switching inspection off to make it go away. The purchase order is signed, the protection is half gone, and nobody wrote that down anywhere.

Test with the ruler you will live with.

SAME BOX · TWO RULERS RFC 3511 · IP THROUGHPUT 1518-byte packets, a handful of sessions, inspection off 40 Gbit/s RFC 9411 · INSPECTED THROUGHPUT HTTPS 64 KB, full handshake, TLS inspection + IPS + App-ID on 6 Gbit/s SIZING GAP 6.7× Buy for the 40, run the 6: the perimeter is under-sized before the first user logs in.

How we run it

RFC 9411 as a service: in CloudLAB or at your site

You bring the DUT or the shortlist; we bring the generator, the method and the report. Scope is agreed before the first packet: which of the 9 tests, which object sizes, which device class, the IPv4 to IPv6 ratio, your traffic mix or the NetSecOPEN one.

Load & Performance Testing

The 9 benchmarks on a generator that fits your setup: Keysight, Spirent or Xena, 1G to 100G. A physical box cabled in Armenia or your virtual NGFW on our hosts, with counters live in your browser.

Request a test plan

Security & Resilience Testing

Appendix A before Section 7: 500 or more CVEs with CVSS 7 to 10, in cleartext and inside TLS, evasions included, with background load running at the same time. Blocked, bypassed, and what the DUT itself reported.

Discuss the CVE set

Test Plan & Methodology Design

A test programme your tender, auditor or vendor can sign: device class, ACL set, traffic mix, cipher suites, pass criteria, report format. Handy when the vendor offers to run the test on their own kit and you want to keep them honest.

Order a test plan

Hosted DUT Testing & Lab Rental

Ship the box or upload the VM, watch the run from a browser, keep the suites. Or rent the bed with the RFC 9411 method already configured and run it with your own engineers.

Log in to CloudLAB
  • Independent: we sell the test, the vendor sells the box
  • Same method and same bed for every candidate on the shortlist
  • Raw results and test suites stay with you

Related reading

FAQ

Questions we get before the first run

The tender still says RFC 3511. Can you test to it?

Yes. RFC 3511 is obsolete, but a procurement document is a procurement document. We run the 3511 set for the paperwork and the 9411 set for the decision, and the report explains why the two numbers differ.

Our NGFW is virtual. Does the method change?

The tests are the same; the report gets longer. Host CPU and RAM, pinned cores, NUMA node, DPDK or SR-IOV, hypervisor version all go in, because a vNGFW result without them cannot be reproduced. We also watch the host for noisy neighbours during sustain.

Do we have to enable TLS inspection?

The RFC recommends it and allows exceptions. If your deployment runs without it, we test without it, state that in the report and add a run with it on, so you see the price of switching it on later.

What if the generator is slower than the DUT?

Then there is no test. The reference pre-test exists for exactly this: the bed has to exceed the DUT's claimed performance before the DUT is connected. We pick the generator and the port count to clear that bar, and the pre-test result is in the report.

Can we bring our own traffic mix?

Yes, and we prefer it. A day of flow data from your perimeter beats any generic mix. We turn it into a profile with the same application shares and object sizes, and test 7.1 runs on it next to the standard one.

Test. Compare. Decide.

Send us the model or the shortlist and the class you are sizing for. You get back a scoped RFC 9411 test plan: tests, object sizes, cipher suites, lab days and price. Then the box goes on the bed, and the datasheet goes in the drawer.

Request a test plan Log in to CloudLAB

Send the shortlist

You have the method. Now get the numbers.

Send the shortlist and the KPIs that matter. An engineer replies within one business day with a test plan draft and a quote.

Prefer to chat? Message an engineer on WhatsApp

We use what you send only to answer you. See our Privacy Policy.