Posts tagged "Benchmarks"

Every dual-DGX-Spark writeup benchmarks bandwidth — GB/s of busbw, ~190 Gbps of RDMA. None of them publish the one figure that actually decides whether a latency-bound multi-node workload is viable: the small-message NCCL all-reduce latency over the ConnectX-7 RoCE link. So I measured it on two GB10 Sparks: ~2 µs RDMA write floor, and ~40 µs for a 16–32 KB all-reduce. Plus the firmware, NCCL, and OpenMPI traps that silently wreck the result.