Tech News
← Home  ·  All topics

Nki Megakernels

1 GoKawiil brief on this topic · RSS for nki megakernels

TrnRT open-source inference engine brings GLM-5.3-Flash to AWS Trainium2

RunanywhereAI released TrnRT, an open-source LLM inference engine built for AWS Trainium chips, launched via a Show HN post. It serves an OpenAI-compatible API from a single trn2.48xlarge instance, using fused NKI 'megakernels' per layer and a prefix cache spanning device and host memory, with GLM-5.3-Flash as its first supported model.