Hugging Face Simplifies Running Models On AWS Trainium
Hugging Face introduces TorchNeuron, a new native backend for PyTorch on AWS Trainium, making it easier to run models on the platform. This development simplifies the process for researchers and developers.
Original reporting · huggingface.co
Original: “TorchNeuron: running HF models on AWS Trainium is easier than ever”
Read “TorchNeuron: running HF models on AWS Trainium is easier than ever” at Hugging Face →
How this was written. Content Delta wrote this article with a language model from Hugging Face’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from Hugging Face’s report
- Hugging Face introduces TorchNeuron, a new native backend for PyTorch on AWS Trainium
- TorchNeuron simplifies the process of running models on Trainium
- The plugin provides support for eager mode, torch.compile, and native distributed APIs
- TorchNeuron enables native distributed training on Trainium
- The plugin supports tensor parallelism for faster model execution on multiple Neuron devices
Hugging Face has announced the introduction of TorchNeuron, a PyTorch plugin that enables native support for AWS Trainium. This move is expected to make it easier for researchers and developers to run models on the platform. TorchNeuron will be available as a public beta starting October 20, 2026.
The Hugging Face experience on AWS Trainium has been delivered through the optimum-neuron library, which relies on the torch-neuronx package. However, this library has some limitations, including lazy tensors and static shapes. TorchNeuron changes this by integrating directly into PyTorch and exposing neuron as a native device.
With TorchNeuron, the Hugging Face libraries, including Transformers, Diffusers, Accelerate, PEFT, and TRL, can run on Trainium directly. This means that researchers and developers can use the same classes and scripts they would use on any other accelerator. The company recommends that Neuron users use these libraries directly instead of going through Optimum Neuron.
TorchNeuron provides support for eager mode, torch.compile, and native distributed APIs. This allows for more efficient and flexible model execution on Trainium. The plugin also enables native distributed training, making it easier to scale up model training on the platform.
In terms of inference, TorchNeuron provides native support for text generation in Transformers. This means that researchers and developers can easily run text generation models on Trainium without requiring significant modifications to their code. The plugin also supports tensor parallelism, which enables faster model execution on multiple Neuron devices.
For training, TorchNeuron allows researchers and developers to use the same Hugging Face libraries as any other accelerator. This means that they can use standard training scripts and configurations, making it easier to train models on Trainium. The plugin also supports torch.compile, which can improve model execution efficiency on the platform.
The introduction of TorchNeuron is expected to simplify the process of running models on AWS Trainium. This can lead to increased adoption of the platform among researchers and developers, and can accelerate the development of AI models and applications.
Questions and answers
What is TorchNeuron?
TorchNeuron is a PyTorch plugin that enables native support for AWS Trainium, making it easier to run models on the platform.
When will TorchNeuron be available?
TorchNeuron will be available as a public beta starting October 20, 2026.
What libraries are supported by TorchNeuron?
TorchNeuron supports the Hugging Face libraries, including Transformers, Diffusers, Accelerate, PEFT, and TRL.
