GoKawiil
Tech News
Google Cloud announced two new TPU systems designed around existing TPU chips, reconfiguring compute, networking, storage and packaging rather than introducing new silicon. The company says the systems respond to customer requests for accelerators that handle large context windows and for specialized encryption features demanded by regulated industries, alongside recent PyTorch support updates.
AI researcher Sebastian Raschka has published an 11-hour, eight-video course accompanying his book 'Build Reasoning Model (From Scratch),' which teaches how to implement reasoning model techniques such as inference scaling, reinforcement learning, and distillation using PyTorch. The course builds on a pre-trained LLM and is positioned as a sequel to his earlier 'Coding LLMs from the Ground Up' course, though it can be watched independently.
Google and Meta jointly released a PyTorch-native version of the SGLang inference engine, called TorchTPU, letting Google's tensor processing units serve AI workloads without requiring a switch to Google's JAX framework. Meta's PyTorch-TPU lead Zongwei Zhou said the engine is already running in production at Meta, built with help from RadixArk, the startup founded by SGLang's creators. A matching PyTorch-native version of the vLLM inference engine was also released as open source.
AMD is folding its ZenDNN library, previously a downstream fork, into the open-source oneDNN project maintained by Intel under the UXL Foundation. AMD also joined UXL as a steering member, paying $20,000 annually for influence over the foundation's direction. The merger was recommended by the x86 Ecosystem Advisory Group, a joint Intel-AMD initiative formed in 2024.
Google announced TorchTPU, a version of the PyTorch framework built to run on its Tensor Processing Units, letting developers shift existing PyTorch-based AI apps to Google Cloud with a simple configuration change rather than a rewrite. The move follows Amazon's release a day earlier of TorchNeuron, its own PyTorch variant for AWS's Trainium chips, and Amazon's submission of code to decouple PyTorch features from Nvidia hardware.
Google has released TorchTPU, an open-source project that lets PyTorch models run directly on its Tensor Processing Units. The package includes custom ATen kernels, a compilation cache for optimizing XLA computations, support for distributed training, and sample implementations for models like Llama, Qwen, ResNet and UNet. It is available via pip as version 0.2.0 or can be built from source using Bazel.
Amazon Web Services released TorchNeuron, a tool that lets developers run PyTorch-based AI applications on its custom Trainium chips by changing a single configuration line rather than rewriting code. AWS says roughly 85% of AI applications are built using PyTorch, the open-source framework now maintained by the Linux Foundation.
PyTorch's activation memory budget solver, which decides which operations in a computation graph to save versus recompute to fit a memory budget, has a new configurable algorithm called dp_knapsack_sliding_hirschberg. Switching the setting from the default dp to this new option reportedly reduces peak RAM usage during memory planning by roughly a factor of 20. The feature is described as a drop-in configuration change via torch._functorch.config.activation_memory_budget_solver.
rGPU is a newly shown project that lets a PyTorch program send tensor operations over TCP to execute on a remote NVIDIA GPU while the application itself keeps running locally. It offers two integration routes: a simple PyTorch 'rgpu' device for programs that opt in, and a CUDA shim that intercepts existing Linux CUDA binaries using libcuda, CUDA Runtime, cuBLAS, cuBLASLt, and cuDNN. Setup involves installing via pip and using an SSH tunnel through the rgpu-run command to connect to the GPU host.
A new open-source repository called OpenArch offers hand-written PyTorch implementations of prominent LLM architectures, including GPT-2 XL, Llama 2, Llama 3, OLMo 2, DeepSeek R1 and Gemma 3. Each model is coded from scratch based on original papers, technical reports and config files, with one clear, self-contained file per architecture rather than production-optimized code.