GPU Benchmarks for LLM Inference: RTX PRO 6000 vs. H200
Performance comparison of the NVIDIA RTX PRO 6000 Blackwell and NVIDIA H200 for LLM inference, based on real-world workloads measured at the MEGWARE Benchmark Center.
A direct comparison of two GPU architectures for modern LLM inference workloads, conducted under controlled and reproducible conditions at the MEGWARE Benchmark Center.
Performance under load
How do response time and throughput change as the number of concurrent requests increases?
Memory & Model Size
What role do memory bandwidth and GPU memory play across different LLM sizes?
Long-Context Inference
How do the GPUs perform with long input contexts and high memory requirements?
GPU Architecture
Which differences between Blackwell and Hopper are relevant for practical inference deployments?
Tested Configurations
Two-GPU servers featuring the NVIDIA RTX PRO 6000 Blackwell and NVIDIA H200 were compared.
NVIDIA RTX PRO 6000 Blackwell
Architecture
Blackwell
Memory
96 GB GDDR7
GPU-to-GPU
PCIe Gen5
NVIDIA H200
Architecture
Hopper
Memory
141 GB HBM3e
GPU-to-GPU
PCIe Gen5 or NVLink
Which Models were tested?
The benchmark report covers different LLM inference requirements – from dense models to a large Mixture-of-Experts model.
Qwen3-32B
A dense model with 32 billion parameters, tested in BF16.
Llama-3.3-70B
A larger dense model with around 70 billion parameters, also tested in BF16.
Qwen3.5-122B-A10B
A MoE model with 125 billion parameters, around 10 billion of which are active, tested in FP8.
Download the GPU Comparison Whitepaper
The complete whitepaper documents the conducted measurements, the used system configurations, and the underlying test methodology in a clear and reproducible manner.
Complete Measurement Results Detailed benchmark results for the RTX PRO 6000 and H200.
Transparent Test Methodology Detailed information on hardware, software, workloads, and the test conditions at the MEGWARE Benchmark Center.
Real-World LLM Workloads Tests covering different model sizes and load scenarios.
