TrnRT open-source inference engine brings GLM-5.3-Flash to AWS Trainium2
RunanywhereAI released TrnRT, an open-source LLM inference engine built for AWS Trainium chips, launched via a Show HN post. It serves an OpenAI-compatible API from a single trn2.48xlarge instance, using fused NKI 'megakernels' per layer and a prefix cache spanning device and host memory, with GLM-5.3-Flash as its first supported model.
GoKawiil's interpretation of the reporting above, not reported fact.
By targeting Trainium instead of Nvidia GPUs, the project could give developers a cheaper, open alternative for serving large models, though real-world cost and performance comparisons aren't detailed in the release. Offline compilation and a radix-tree prefix cache suggest an engineering focus on reducing runtime overhead and serving long-context workloads efficiently, which could matter for teams running inference at scale on AWS hardware.
- TrnRT is an open-source inference engine built specifically for AWS Trainium2 hardware.
- It uses custom NKI megakernels per model layer and supports up to 1M-token context via sparse/linear attention.
- GLM-5.3-Flash is the first supported model, with more open models planned.
Source: github.com, 2026-11-17
Published there as: “Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.