GoKawiilTesting shows 8x RTX PRO 6000 rigs excel at parallel serving, not giant model sharding
Jerry James benchmarked an 8-GPU NVIDIA RTX PRO 6000 Blackwell system paired with an AMD EPYC 9555 CPU, offering 768GB of combined GDDR7 memory. Rather than splitting huge models like Llama-3.1 405B or DeepSeek-R1 671B across all eight PCIe cards, which introduces heavy latency, the team found the setup performs best running independent single-GPU model instances in parallel.