We test the network side of storage: NAS and SAN over Ethernet with NFS, SMB, iSCSI and NVMe/TCP, NVMe-oF over RoCEv2 on lossless Ethernet, S3-compatible object storage such as RustFS, hyperconverged clusters under node failure, backup and replication windows over an impaired WAN, and the top-of-rack switch that carries all of it next to the application traffic. Ports at 1G, 10G and 100G in CloudLAB, 25G as breakout from 100G; your array, your hosts or your virtual appliance in the rack or on the lab hypervisor.
When the number is needed
A datasheet gives the throughput of an array on a direct cable with one block size. Each moment below asks what the array, the hosts and the switch between them do with your block sizes, your mix and your failures, and ends with a number in a report.
Every candidate takes the same I/O profile from the same hosts through the same switch: block sizes, read and write mix, queue depth, protocol. Vendors see the plan before the first I/O and the raw results after the last one.
Ranked table per KPI, ceiling per model, 3–4 weeks for 3 systems
The delivered array, the planned ToR pair and your host profile with the production config: MTU, PFC and ECN per queue, multipathing, NIC offloads, bonding. Measured against the design, with node, link and path failures during the run.
Signed report with pass and fail per case, config exports, from 2 weeks
The acceptance cases run again on the new array firmware, the new NIC driver, the new switch release or the changed QoS map. What moved, by how much, and whether the backup window still closes before the morning.
Diff per KPI against the last run, 3 business days
What we test
The hosts, the switch and the I/O profile stay the same across the row; the pass criteria do not. The table is the starting point of the plan; your workload and your design set the thresholds.
| Device class | Numbers in the report | Traffic and config on the bench | Typical size |
|---|---|---|---|
| 01NAS and SAN array with an Ethernet front end | Throughput and IOPS per block size and queue depth, latency p50 and p99, controller failover | Bench hosts drive your I/O profile through the ToR pair, app mix on the same ports, multipathing | 2–8 front-end ports, 25G and 100G |
| 02NVMe-oF target and initiator pair over RoCEv2 or TCP | Throughput and IOPS at 4 kB to 1 MB, latency per queue, pause time and ECN marks per port | Initiators on bench hosts, PFC and ECN on the storage queue, incast on one target, a link failed | 100G ports, 8–32 initiators |
| 03Hyperconverged and S3 object storage cluster | S3 PUT and GET throughput and latency per object size, rebuild time on node loss, east-west traffic | Nodes in the rack or on the lab hypervisor, VMs or S3 clients, objects 1 kB to 128 MB, GET, PUT and mixed | 3–8 nodes, 25G uplinks |
| 04Top-of-rack or leaf switch carrying storage | Loss and pause per queue with storage and app traffic, buffer headroom on incast, uplink failover | Storage flows and the app mix at line rate, your QoS, PFC and ECN config, RFC 2544 baseline | 32–64 ports, 25G and 100G |
| 05Backup server and WAN link | Backup and replication window over your WAN profile, throughput per stream, restore time of a sample | WAN emulator with your delay and loss, the backup product on the bench, jobs at peak and on failover | 1G to 10G WAN, 10 to 200 TB sets |
Any vendor, physical or virtual, including software-defined storage on bench hosts. Named test tools are picked for the setup and listed in the report. Backup and VoIP products we also integrate and support in client networks after the test.
Around the array
An IOPS number only means something with the right things around the array: the hosts that will drive it, the switch and the queues it will cross, and the failures it has to survive with I/O running
PFC per queue, ECN thresholds, DCBX, MTU 9000 and the QoS map from your export. The bench counts pause frames, ECN marks and drops per port, so a slow array can be told apart from a switch that pauses the whole link.
4 kB random, 64 kB and 1 MB sequential, the read and write mix and the queue depth from your workload statistics, driven from as many hosts as your design has. For object storage: GET, PUT and mixed with object sizes from 1 kB to 128 MB, distributed clients, latency percentiles. Not one stream from one laptop.
Node loss, uplink loss, NIC port failover, controller and path failover, a backup job started during peak. Each one timed from the last completed I/O to the first one back, with latency recorded through the event.
Typical scenario
An integrator brings a new array and the planned ToR pair for acceptance before a migration. On a direct cable the array shows the datasheet figure. Through the ToR with the delivered config and the application mix on the same 100G ports it gives 6.8 GB/s with 0.3 % loss: no PFC on the storage queue, ECN off, incast from 16 hosts into one port. With PFC on that queue, ECN thresholds set and storage in its own class, 10.4 GB/s and no loss, with the pause frames and the ECN marks counted per port.
Composite scenario from projects of this type. Numbers are realistic and not from one client.
What you take away
The report is built for the meeting after the test: the array vendor, the switch vendor, the integrator and your platform team read the same table, with the pass band from the design next to the measured value
Each number comes with its evidence: host-side I/O statistics, per-queue counters from the switch, array counters, captures and the config export of the run
| KPI | Pass band | Measured | Result |
|---|---|---|---|
| Sequential read, 1 MB, QD 32, 16 hosts | ≥ 10 GB/s | 10.4 GB/s | Pass |
| Random read, 4 kB, QD 64, IOPS | ≥ 1.5M | 1.7M | Pass |
| Latency p99, 4 kB random, QD 64 | ≤ 300 µs | 240 µs | Pass |
| Frame loss at 100G with the application mix | 0 | 0 | Pass |
| Controller failover, I/O stall | ≤ 5 s | 2.4 s | Pass |
| Backup window, 20 TB over 10G WAN, 30 ms RTT | ≤ 9 h | 8.6 h | Pass with a note |
Where it runs
Arrays and switches ship to CloudLAB or we connect the bench to your lab; virtual appliances and software-defined storage run on the bench hypervisor, and you watch the run through the portal. The bench has 1G, 10G and 100G ports, 25G as breakout from 100G, 100G ports for RoCEv2 agreed per setup, WAN emulators per link, rack space and VMs for your systems, KVM and serial on every unit, and a TEST4NET engineer on every session.
How a load run goes, step by step AI fabric and RoCEv2: lossless Ethernet at scale Cloud and NFV: virtual appliances under load Hosted DUT: your system stays in the rack
Before the brief
Any vendor with an Ethernet front end: NAS and SAN arrays over NFS, SMB, iSCSI and NVMe/TCP, NVMe-oF targets over RoCEv2, S3-compatible object storage such as RustFS, hyperconverged clusters, software-defined storage on bench hosts, backup servers and their WAN. The bench is Ethernet.
No data. The I/O profile comes from your workload statistics: block sizes, read and write mix, queue depth, host count, object sizes for S3. Bench hosts drive it; your own servers can sit in the rack instead if the NIC, the driver or the bonding is part of the question. A data-warehouse workload modelled on TPC-DS, on a query engine over S3, is agreed per project and reported as our own run, not as a TPC result. Configs come as anonymised exports.
PFC, ECN and DCBX are tested per queue on the switch under test, with pause time and marks counted per port and incast driven from 8 to 32 initiators. For a leaf-spine fabric carrying RDMA at scale see the AI fabric and RoCEv2 page; the storage page covers the array, its targets and the ToR in front of them.
Yes. A WAN emulator sits on the link with your delay, jitter and loss, the backup product and its agents run on the bench, and the job is timed for a sample set and extrapolated to yours. The same run repeats after a link failover and during peak application traffic.
Yes. Arrays, nodes and switches go into the CloudLAB rack with KVM and console access and return after the run, or stay cabled for the next firmware under a hosted plan. For a system that is already racked we connect the bench to your lab over a dedicated link.
Test plan and quote in 1–3 business days after the brief. A run takes from 2 weeks for one array or one cluster and 3–4 weeks for a shortlist of 3 systems. The report has every case with its pass band, the measured value, per-queue counters, captures and config exports, plus a one-page summary for the steering meeting.
No prices on the site: scope first, then the quote. The array vendor and the integrator can sit in on the brief.
Send the shortlist
Name the models and the KPIs that matter. An engineer replies within one business day with a test plan draft and a quote.
Prefer to chat? Message an engineer on WhatsApp
We use what you send only to answer you. See our Privacy Policy.