Hugging Face Introduces Model Support Space For AWS Trainium
Hugging Face launched a model support space to help users determine which models run on AWS Trainium, according to Hugging Face. The space provides information on model compatibility and performance.
Original reporting · huggingface.co
Original: “Does Your Model Run on AWS Trainium? Look It Up”
Read “Does Your Model Run on AWS Trainium? Look It Up” at Hugging Face →
How this was written. Content Delta wrote this article with a language model from Hugging Face’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from Hugging Face’s report
- Hugging Face launched a model support space to help users find out which models run on AWS Trainium.
- The space provides information on model compatibility and performance, including the number of Trainium cores and peak device memory.
- Users can select a model to see how to run it on Trainium and access the run.py script and Neuron stack used for testing.
- The space currently only tests Transformers models, but plans to expand to other models in the future.
- The test results are available as a dataset that can be queried directly using pandas.
Hugging Face introduced TorchNeuron, a native PyTorch backend for AWS Trainium, allowing Transformers models to run on the device. The company has now launched a model support space to help users find out which models are compatible with Trainium.
The model support space, called the torch-neuronx model support Space, provides information on the most popular models on the Hub. It tells users whether a model runs unmodified on Trainium and gives them the script to run it themselves, according to Hugging Face.
The Space has three views: Selected, Trending, and Downloads. The Selected view shows a curated list of models picked from the top models on the Hub by downloads and trending score. The Trending and Downloads views follow the Hub's own rankings and allow users to search for specific models by name.
When a user looks up a model, the Space provides information on the result, the number of Trainium cores the model is sharded over, the wall time of the run, and the peak device memory. It also explains why a model was not tested, if applicable.
Users can select a model to see how to run it on Trainium. The Space lists the Neuron stack the model was tested on, the packages to install, and the run.py script that ran the model. The script follows the recipe of the TorchNeuron announcement and loads the model with from_pretrained, sharded when necessary.
The model support space currently only tests Transformers models, but Hugging Face plans to expand to other models, including diffusers, sentence transformers, and other transformers tasks. The space also provides a dataset of the test results, which can be queried directly using pandas.
The dataset contains information on each model's result, error messages, and versions of the whole stack and run.py script. The dataset card provides the exact meaning of each result, allowing users to understand the performance of their models on Trainium.
Questions and answers
What is the torch-neuronx model support Space?
The torch-neuronx model support Space is a tool that helps users determine which models run on AWS Trainium, according to Hugging Face.
How do I run a model on Trainium?
To run a model on Trainium, select the model in the Space and follow the instructions to install the necessary packages and run the run.py script.
What information is available in the dataset?
The dataset contains information on each model's result, error messages, and versions of the whole stack and run.py script, which can be queried directly using pandas.
