#Scaling LLM Test-Time Compute Optimally

The traditional paradigm of scaling model intelligence has focused almost exclusively on the pre-training phase, treating performance as a static outcome of parameter count and training data volume. This pre-training centric view assumes that a model's reasoning capability is "frozen" at the point of deployment, requiring ever-larger models to solve increasingly complex problems. However, human cognition suggests a more dynamic approach, where the amount of effort expended is proportional to the difficulty of the task. The exploration of "test-time" or inference-time compute as a new scaling frontier suggests that the intelligence of a system can be expanded during the generation process itself, allowing a smaller model to overcome its inherent knowledge gaps through iterative search and refinement.

#Revisions and Verifier-Guided Search

Comparing different PRM search methods: Best-of-N, Beam Search, and Lookahead Search.

The optimization of inference-time compute utilizes two primary mechanisms: sequential revisions and verifier-guided search. Sequential revisions involve fine-tuning a model to critique its own previous outputs and iteratively correct local logical errors, effectively conditioning its generation on its own history of mistakes. In parallel, verifier-guided search—utilizing process-based reward models (PRMs)—evaluates intermediate reasoning steps to prune incorrect paths and focus computational resources on the most promising solution trajectories. This finding revealed that for easy tasks, the model is likely "on the right track" and merely requires the local refinement of a revision cycle, whereas difficult problems necessitate the broader global exploration of a tree-search algorithm.