Tech News
← Home  ·  All topics

Tensor Parallelism

1 GoKawiil brief on this topic

NVIDIA demos GLM-5.3 served at 607.8 tok/s across two DGX Stations with vLLM

NVIDIA published a tutorial showing how to serve Z.ai's GLM-5.3 NVFP4 model, a 753-billion-parameter mixture-of-experts model with 40 billion parameters activated per token, across two GB300 DGX Stations linked via ConnectX-8 RDMA. Because the roughly 464 GB quantized checkpoint exceeds the 252 GB of HBM3e on a single GB300 GPU, the setup uses tensor parallelism (TP=2) and vLLM to split weights and computation across both machines while exposing one OpenAI-compatible API endpoint, reportedly measuring throughput of 607.8 tokens per second.