Skip to main navigation Skip to main content Skip to page footer
megware_com logo megware_com logo

GPU-Interconnect Benchmarks for LLM-Inference:
NVLink vs PCIe Switch vs PCIe Direct

Performance comparison of three GPU interconnect architectures for four NVIDIA H200 NVL GPUs,
based on real-world LLM inference workloads measured at the MEGWARE Benchmark Center.

Download Free Whitepaper   Get your Free Test Drive

What to expect from this Interconnect Comparison

NVLink vs. PCIe

How much does the connection between the GPUs affect LLM inference performance?

Performance under Load

How do response time and throughput change with up to 64 concurrent requests?

Long-Context-Workloads

How do NVLink and PCIe Direct perform with long prompts of up to 128,000 tokens?

Tested Configurations

Three different GPU interconnect architectures were compared in a server with four NVIDIA H200 NVL

NVLink

Dedicated full-mesh connection between all four GPUs

PCIe Switch

Four GPUs connected through a shared PCIe switch

PCIe Direct

Direct PCIe connections from the GPUs to the CPU

Hardware: 4× NVIDIA H200 NVL, Dual-Socket AMD EPYC 9654
Software: vLLM, Tensor Parallelism (TP=4).

What was evaluated in the Benchmark Report?

Interconnect Bandwidth

How do the three architectures differ in GPU-to-GPU communication and All-Reduce operations?

Response Time & Throughput

How does the interconnect affect LLM response time and throughput as the workload increases? 

Long-Context Inference

How do the different interconnects perform with long input contexts?
 

Download the Interconnect Comparison Whitepaper

The MEGWARE Benchmark Center whitepaper documents the tested GPU interconnect architectures, 
the LLMs used, detailed benchmark results, and the underlying test methodology.

Complete Benchmark Results
Detailed results for NVLink, PCIe Switch, and PCIe Direct.

 

Real-World LLM Workloads
Tests with Qwen3-32B, Llama-3.3-70B, and Qwen3.8-Flash-Next-FP8.

 

Transparent Test Methodology
Detailed information on hardware, software, workloads, and test conditions.

GPU Interconnect Benchmark Report Whitepaper

I accept the privacy policy and consent to the processing of my data.