How we test multi-vendor interop: one RFC, two readings

Two routers from two vendors, and both datasheets list the same RFCs. Both are telling the truth. An RFC leaves options open and every vendor picks its own defaults, so two compliant boxes can bring BGP up, show Established on both consoles and still exchange half the routes. We test the pair: your device against the implementations it will meet, with your configuration on both ends and traffic across the boundary.

The bench

Your device goes onto the CloudLAB bench, or runs as a virtual appliance on our hosts. Next to it, cabled to the same 1G to 100G ports, sits the second vendor: a real box from the lab, a box you ship, or its virtual edition. The bench generators add emulated peers of several implementations, so one device role meets up to 4 neighbours in a single interoperability project.

The pair list comes from your network diagram: who peers with whom, over which protocol, with which template. Both ends run the firmware you will deploy and your production timers, authentication and policy. Traffic crosses the boundary in both directions, so every result has a forwarding number next to the session state.

What we check

• Capability negotiation: address families, 4-byte AS, add-path, graceful restart; what each side offered, what was agreed, routes sent against received per family
• Policy across the boundary: communities, local preference and MED set by one vendor and read by the other, every attribute checked on arrival
• Defaults that differ: hello, hold and BFD timers, MTU on the link and inside the overlay, run once with your template values and once with each vendor's own
• EVPN between vendors: route types 2, 3 and 5, the type 5 model each side uses, ESI handling and VNI to VRF mapping
• MPLS and segment routing: label exchange, label ranges and SID conflicts, stack depth, fast reroute from one implementation to the other
• OAM: Y.1731 MEPs of two vendors on one MEG, CCM interval and level, RDI and AIS crossing in both directions, delay and loss set against the generator
• Failover: a link pulled between the vendors, a node reloaded, graceful restart in helper mode; the forwarding gap per flow and per direction, in milliseconds
• Negative cases: an unknown capability, an unexpected attribute, mismatched MTU or authentication; does the far side reset the session, ignore the option or drop routes silently

Protocol detail is in our post on EVPN and VXLAN; here every item runs between two implementations, and every fail is repeated with captures on both ends of the cable.

What usually breaks

Routes that never arrive. A typical core migration: a new router model replaces half of the core. BGP comes up, 1M routes load, and the new side silently drops a community set by the old one. Downstream policy keyed on it sends traffic the wrong way. The bench catches it in the policy pass; one configuration line fixes it before the cutover.

One number, two meanings. One vendor writes MTU 1500 and means the IP packet, the other writes 1514 and means the Ethernet frame. OSPF sits in ExStart, or the adjacency comes up and the first full-size packet across the VXLAN overlay is lost. Timers repeat the story: CCM at 3.3 ms on one end against 10 ms on the other raises an unexpected-period alarm on a healthy link.

Failover that belongs to neither side. A link between the vendors is pulled. One side runs BFD at 50 ms, the other accepts nothing below 300 ms, and the session settles on the slower value: in a typical project about 900 ms to detect against 150 ms in the design. The gap in milliseconds is a property of the pair, so we measure it per pair.

What is in the report

Page 1 is the interop matrix: pairs down the side, implementations across, a pass, a fail or a pass with a note in each cell. Behind every cell sits a record: both configurations as they ran, a capture from each end per fail, routes received, sessions up and the convergence gap per run. The fix list says what to change, on which side, and in which release the vendor reports it fixed. When two vendors blame each other, the captures show who sent what; on request we walk both through them on one call.

Where to start

Send us the network diagram, the device list of each vendor and the configuration templates. We answer with a draft test plan: the pairs, the bench layout and the dates. The report is yours to keep and to show to both vendors. Write to us or look at what else the bench does.