Does the network decide time to first token?
The KV Cache Benchmark splits prefill and decode across two Dell PowerEdge XE9680 servers (16 AMD Instinct MI300X
GPUs), moves the KV cache over 1 GbE, a Broadcom 400G RoCE fabric or 400G TCP, and shows what each path does to
time to first token and throughput.
The interactive dashboard needs JavaScript. Please enable it to explore the results.
Benchmarks by Metrum AI. Copyright © 2026 Metrum AI. All rights reserved.