Share on Bluesky Share on Mastadon Share on LinkedIn Share on Twitter/X Share on Reddit Share on Facebook Share on Whatsapp Share via Email
If you’re interested in deploying advanced AI models on your local hardware, leveraging a modern AMD GPU or APU can provide an efficient and scalable solution. You don’t need dedicated AI infrastructure to experiment with Large Language Model (LLMs); a capable Microsoft® Windows® PC with PyTorch installed and equipped with a recent AMD graphics card is all you need.
PyTorch for AMD on Windows and Linux is now available as a public preview. You can now use native PyTorch for AI inference on AMD Radeon™ RX 7000 and 9000 series GPUs and select AMD Ryzen™ AI 300 and AI Max APUs, enabling seamless AI workload execution on AMD hardware in Windows without any need for workarounds or dual-boot configurations. If you are new and just getting started with AMD ROCm™, be sure to check out our getting started guides here.
This guide is designed for developers seeking to set up, configure, and execute LLMs locally on a Windows PC using PyTorch with an AMD GPU or APU. No previous experience with PyTorch or deep learning frameworks is needed.
What you’ll need (the prerequisites)
The currently supported AMD platforms and hardware for PyTorch on Windows are listed here:
AMD Radeon™ AI PRO R9700 AMD Radeon™ RX 7900 XTX AMD Radeon™ PRO W7900 AMD Ryzen™ AI Max+ 395 AMD Radeon™ RX 9070 XT AMD Radeon™ PRO W7900 Dual Slot AMD Ryzen™ AI Max 390 AMD Radeon™ RX 9070 AMD Ryzen™ AI Max 385 AMD Radeon™ RX 9060 XT AMD Ryzen™ AI 9 HX 375 AMD Ryzen™ AI 9 HX 370 AMD Ryzen™ AI 9 365
Part 1: Setting up your workspace
Step 1: Open the Command Prompt
First, we need to open the Command Prompt
... continue reading