A router datasheet says "BGP: 4M routes, 1,000 peers". The number is true in the sense that the table fits in memory. It says nothing about how long the box takes to install those routes, what happens to forwarding while it does, and how many seconds of blackholing you get when one upstream drops. Those 3 things are what we measure.
The bench
The device under test sits in the rack with 1G, 10G or 100G ports cabled to the traffic generator. The generator emulates the neighbours: eBGP upstreams, iBGP route reflectors, a few hundred peers if the design calls for it, with IPv4 and IPv6 address families, VPNv4 for MPLS designs, EVPN where the box is a fabric leaf. Each emulated peer advertises its own slice of the table, up to 1M routes in total, and the same generator sends traffic to those prefixes so we see forwarding as well as the RIB.
The configuration is yours. We take the policy from the production template: communities, local preference, AS-path filters, prefix limits, the route maps the operations team actually wrote. A test on a blank config tells you about the chip. A test on your config tells you about your network.
What we check
- Session setup: 100, 500 and more peers brought up at once, with MD5 or TCP-AO where the design uses them, and the time until every session is Established
- Table load: 1M routes advertised at once, time until the last prefix is in the FIB, CPU and memory during the load
- Policy: each route map and prefix list fed with routes that should pass and routes that should fail, the result read back from the generator side
- Convergence: a peer goes down, a link is pulled, a prefix is withdrawn; the generator times the gap in forwarding for every affected flow, in milliseconds
- Churn: continuous flaps of a few thousand prefixes for an hour, with the forwarding plane under load the whole time
- Limits: max prefix limits, graceful restart and BFD timers, route dampening if it is enabled
- Soak: 24 hours at 80% of the measured capacity, with the route table and the traffic running, then a check that nothing leaked
The list is 80 to 150 items depending on the design. Each item has a pass criterion written before the test starts, so the result is a yes or a no, not an opinion.
What usually breaks
The table loads fine. Convergence is where the surprises live. A typical case: a box with a 1M-route table takes 40 seconds to withdraw 300,000 prefixes after an upstream fails, and for those 40 seconds the traffic to the lost prefixes is forwarded into a dead interface. The datasheet had no line for that.
Second case: policy that works for 10 prefixes and quietly stops working at scale. A prefix list with a few thousand entries, a route map with a continue clause, and a share of routes pass that should have been dropped. Nobody sees it in production until a customer announces something odd.
Third case: BFD at 50 ms intervals holds with 10 sessions and starts flapping at 200, because the control plane is busy installing routes at the same time. The fix is a timer, but you want to know the number before the rollout, not after the first outage.
What is in the report
Every item with its criterion and its result. Convergence as a table: event, number of affected prefixes, gap in milliseconds, worst flow and median flow. Table load as a curve of routes installed over time. CPU and memory charts for the whole run. The configuration of the device and of the emulated peers, so the vendor can reproduce any item. Firmware version, port speeds, the bench layout. Nothing in the report depends on which generator we used.
Where to start
Send us the device model, the firmware you plan to run and the topology it will sit in. We answer with a test plan: the items, the bench layout and the dates. The plan is free, the report is yours to keep and to show to the vendor. Write to us or look at what else the bench does.
Alexander Zemskov
4 min read
