Skip to content

Every feature on, threats on the wire: the number you can plan with

We put your NGFW, IPS or VPN gateway between emulated users and servers in CloudLAB, switch on the features you will run in production (IPS, application control, TLS inspection, logging) and send application traffic with exploits, malware and evasions mixed in. One run answers 2 questions at once: how much the device blocks and how much traffic it still passes, at up to 200 Gbps, through policy limits, an HA failover and an upgrade under load. A run takes from 2 weeks. DDoS testing is out of scope. The test plan and the quote come within 1–3 business days.

Request a test plan See the 7 test areas
One NGFW, 40 Gbps on the datasheet Throughput as features go on. Representative result.
Datasheet Firewall only + IPS + TLS inspection + threats in the mix 40 Gbps 38 18 9 6 Gbps Block rate at idle 97.4% Block rate under load 91.2%
  • 200 Gbps+L4-7 and security bench
  • 2 answersefficacy and performance in one run
  • From 2 weeksfor a run
  • 1–3 daystest plan and quote
The same device, 3 measurements
010203040 85%90%95%100% Throughput, Gbps Block rate Pass criterion 95% Capacity plan 20 Gbps Datasheet 40 Gbps · 99% Idle bench 36 Gbps · 97.4% Under load, with evasions 6 Gbps · 91.2%

Where datasheets stop

Three things a security datasheet never says

The throughput line is measured with one feature on, clean traffic and a steady state. Your perimeter runs 6 features, carries attacks and loses a link once a quarter.

  1. Features are measured one at a time

    Firewall throughput 40 Gbps, threat prevention 18 Gbps, TLS inspection 9 Gbps: 3 lines, 3 separate runs. Nobody measures the 3 together, and the 3 together is what you deploy.

  2. The attack set is the one the vendor practised on

    A box tuned for the test tool sample set blocks 99% of it. Change the evasion (fragment, chunk, gzip, TLS 1.3) and the same exploit walks through. We run each attack both ways and report both numbers.

  3. Failover is timed on an idle box

    An HA pair switches in 1 s on an empty bench. With 2 million sessions and a full log queue it takes 14 s and drops a third of the sessions. The datasheet line was true at idle.

What goes into the plan

Seven areas, two numbers each: what it blocks and what it passes

The plan starts from the role of the device (perimeter, data centre, branch VPN, inspection in front of a web farm), your policy and the features you will run. Each area has its own traffic, its own attack set and a pass criterion you sign before the run. Nothing changes afterwards without a note in the report.

One area, one timeline: application load, attacks in the mix, one failover
Init Ramp up, 60 s Sustain, 300 s Ramp down Collection Attacks every 30 s HA failover at 200 s KPIs sampled every 2 s Application traffic
  • Ramp up, 60 s
  • Sustain, 300 s
  • Attacks every 30 s
  • HA failover at 200 s
  • KPIs sampled every 2 s
AreaWhat we runWhat we measureEvidence in the report
01Threat efficacy Exploits for published CVEs, malware samples and C2 callbacks, mixed into live application sessions Block rate per category, every miss with its attack ID Captures of each miss, device logs
02Evasions The same exploits with IP fragmentation, TCP segmentation, HTTP chunking and encoding, TLS 1.3 Block rate with and without each evasion A pair of captures per technique
03TLS inspection TLS 1.2 and 1.3, your cipher list, certificates from your CA, pinning and bypass lists Decrypted throughput, connections per second, handshake latency Per-cipher table, certificate errors seen by clients
04Application performance RFC 9411 mix or your own AppMix, HTTP and HTTPS connections per second, concurrent connections, latency Throughput with every feature on, the 9 RFC 9411 KPIs KPI samples every 2 s, the full load profile
05Policy and objects at scale 5,000 rules, 50,000 objects, NAT, user and group identity, URL categories Throughput and latency against policy size, rule lookup time Policy sets, measured curves
06VPN IPsec and SSL tunnels from 1 to 5,000, rekey, fragmentation, mixed cipher suites Throughput per tunnel and total, tunnel setup rate, loss at rekey Tunnel tables, counters, captures
07Resilience HA failover under load, link loss, session sync, upgrade with traffic, log storm, management plane under attack Failover time, sessions kept, time back to a clean state Timeline and loss per event

DDoS and volumetric flood tests are not part of this service: over public links they are a legal risk, and we do not run them on the bench either. Resilience here means failover, link loss, upgrades and log storms under load.

How a run goes

Five steps, every miss on the day it appears

  1. 1

    Brief

    1 call

    The device, its role, the policy, the features you will run and what blocked means for you

    Test plan and quote in 1–3 business days

  2. 2

    Plan

    Days 1–3

    Traffic mix, attack set, evasions and the 7 areas with a pass criterion each. You sign it off.

    Signed plan, the only scope of the run

  3. 3

    Bench

    Days 3–5

    Your appliance in the rack or your virtual image on our hosts, your policy loaded, certificates issued, the reference pre-test without the device

    Topology, baseline config, pre-test numbers

  4. 4

    Execution

    Days 5–12

    Area by area, 300 s of sustain each, every miss captured. A blocker goes to you and into the vendor case the same day.

    Daily status, misses as they appear

  5. 5

    Report and retest

    Days 12–14

    Block rate and throughput per area, every miss with its capture, the failover timeline. A retest after a signature or firmware update takes up to 3 days.

    Report, evidence archive, retest option

Typical scenario

Two NGFW candidates, one 40 Gbps perimeter, one of them keeps its promise

A bank replaces the perimeter pair for a 40 Gbps uplink and shortlists 2 vendors. Both datasheets say 40 Gbps of threat prevention. On the bench, with IPS, application control and TLS inspection on and attacks in the mix, vendor A holds 21 Gbps and blocks 96.1%, vendor B 9 Gbps and 89.4%. Failover with 2 million sessions: 1.8 s against 14 s with 31% of the sessions dropped. The bank buys vendor A one size up and writes 21 Gbps into the capacity plan instead of 40.

21 vs 9 Gbps
with every feature on
96.1 vs 89.4%
blocked under load
1.8 s vs 14 s
HA failover with 2 M sessions

Composite scenario from projects of this type. Numbers are realistic and not from one client.

Vendor A Vendor B 40 Gbps on both datasheets Firewall only 38 36 + IPS 29 19 + TLS inspection 23 11 + threats in the mix 21 Gbps 9 Gbps

The evidence

What a miss looks like in the report

Every attack that gets through has the same record: what it was, how it was sent, what the device logged and what we captured on both sides. The vendor can reproduce it, and your SOC can check whether the production box behaves the same way.

Four rules decide what counts, agreed in the plan before the first packet

Blocked
The session is reset or dropped and the device logs the event within 5 s. A silent drop without a log entry counts as blocked, with a note.
Missed
The payload reaches the server side. Severity by category: exploit, malware, C2 or evasion only.
False positive
Clean traffic from the application mix is blocked. Counted per 10,000 transactions.
Degraded
The device keeps blocking, but a KPI leaves the pass band: latency over the limit, failed transactions over 0.001%, CPU over 90% for 60 s
Miss 112 Evasions Missed

Web server exploit sent with HTTP chunked encoding and a gzip body

Sent
The exploit from item 37 of the efficacy set, chunked transfer encoding, gzip body, over TLS 1.3, at 60% of the measured throughput
Observed
Blocked at idle. At 60% load the payload reached the emulated server in 4 of 5 trials, and the device logged nothing.
Verdict
Missed. Severity: major for a perimeter with public web servers behind it.
Evidence
Captures 112-a (client side) and 112-b (server side), device log excerpt, IPS profile export
Vendor case
Opened on day 7 of the run, signature update promised, retest scheduled

Where it runs

Emulated users on one side, servers and attacks on the other

The bench generates application traffic and attacks from both sides of your device at 200 Gbps and more, with TLS 1.3 and current exploit, malware and evasion sets. Your appliance goes into the rack with KVM and console access, or your virtual image runs on our hosts. You watch the run in the browser. The generator and the attack library are picked for the setup and named in the report.

Bench access and capacity Open the CloudLAB portal RFC 9411: the firewall benchmark we run Combine with load and performance testing

Related reading

Questions before the brief

What buyers ask first

Do you test with our policy or with the vendor defaults?

With yours. You export the policy, we load it and switch on the features you run in production. Where you have none yet, we use the vendor profile recommended by RFC 9411 and say so in the report.

Where do the attacks come from and how current are they?

From the exploit, malware and evasion sets of the bench generators: published CVEs of the last years plus the classic evasions. The plan lists the set by category and count, and you can add samples from your own incidents.

Can you inspect TLS on the bench the way we do in production?

Yes. The emulated clients and servers use certificates from your CA or ours, TLS 1.2 and 1.3 and your cipher list. Inspected throughput and handshake latency come out per cipher suite.

Can you compare 2 or 3 candidates side by side?

That is the most common form of this service. Same policy, same traffic, same attack set, same week, and the report puts the candidates in one table per area. Vendors ship evaluation units under their own loan terms.

Do you run DDoS tests?

No. Volumetric tests over public links are a legal risk, and we do not offer them on the bench either. Resilience in this service means HA failover, link loss, upgrades and log storms under load.

How long does a run take, and what happens when the box fails?

From 2 weeks for the 7 areas, including the plan and the report. A miss or a blocker is reported on the day with the capture, so the vendor case opens while the run continues. A retest after a signature or firmware update takes up to 3 days.

Send the device model and your policy. We send back the plan.

Test plan and quote in 1–3 business days. No box on hand yet is fine: tell us the candidates and the uplink.

Request a test plan All services

Send the shortlist

You have the method. Now get the numbers.

Name the models and the KPIs that matter. An engineer replies within one business day with a test plan draft and a quote.

Prefer to chat? Message an engineer on WhatsApp

We use what you send only to answer you. See our Privacy Policy.