S1 test-time scaling explained: budget forcing and benchmark results
Quote514 words
Explains S1's budget forcing, which suppresses end-of-thinking to append Wait or forces early termination, so extra inference compute improves answers. Trained on 1,000 curated samples, S1-32B beats o1-preview on AIME24 and MATH500.
Key claims
AIME24 57.0% vs o1-preview 44.6%
MATH500 93.0% vs 85.5%
GPQA Diamond 59.6% vs 73.3%
26 minutes of training on 16 H100s
S1 Test-Time Scaling: Simple But Powerful
S1 represents a disruptive approach to language model reasoning that achieves impressive results with minimal training resources through an elegant technique called budget forcing. Unlike traditional models that rely on extensive pre-training and massive datasets, S1 leverages a simple yet powerful mechanism to dynamically control and extend the reasoning process during inference time.
➊ How Budget Forcing Works
Budget forcing is a sequential test-time scaling method that intervenes in the model's decoding process in two ways:
➤ Extending Reasoning: When the model attempts to conclude its reasoning prematurely, the system prevents it from stopping by suppressing the end-of-thinking token and appending "Wait" to the current reasoning trace. This prompts the model to continue its analysis, often leading to self-correction and improved answers.
➤ Limiting Reasoning: Conversely, if a reasoning trace becomes too long, the system can force early termination by appending the end-of-thinking token and "Final Answer:" to make the model provide its current best solution.
This approach allows S1 to achieve substantial performance improvements by simply spending more compute during inference rather than during training.
➋ Traditional LLMs vs. S1's Approach
Traditional LLM Inference
➝ Fixed computational path for all inputs regardless of complexity
➝ Uniform processing where all layers handle every token
➝ Predetermined stopping criteria with no opportunity for refinement
➝ Relies heavily on extensive training data and pre-training compute
S1 Test-Time Scaling
➝ Dynamic control over reasoning length through budget forcing
➝ Ability to trigger additional thinking for self-correction
➝ Trained on just 1,000 carefully selected high-quality reasoning samples
➝ Extremely sample-efficient (26 minutes of training on 16 H100 GPUs)
Sample Efficiency: Quality Over Quantity
S1's impressive performance stems from its focus on quality over quantity in training data. The researchers curated a dataset (s1K) of just 1,000 examples based on three principles:
➝ Quality: High-quality, well-formatted examples from diverse sources
➝ Difficulty: Problems that are challenging enough to require significant reasoning
➝ Diversity: Broad coverage across 50 different domains of maths & science
This approach stands in stark contrast to traditional models and even OpenAI's o1, which likely used millions of examples and extensive reinforcement learning.
➌ Benchmark Performance
Despite its minimal training, S1-32B with budget forcing achieves remarkable results on challenging benchmarks:
➝ AIME24: 57.0% (compared to o1-preview's 44.6%)
➝ MATH500: 93.0% (compared to o1-preview's 85.5%)
➝ GPQA Diamond: 59.6% (compared to o1-preview's 73.3%)
S1 outperforms o1-preview on AIME24 by over 12 percentage points and on MATH500 by 7.5 percentage points, while using orders of magnitude less training data and compute.
➍ Why This Matters
S1 demonstrates that test-time scaling offers a new paradigm for improving AI reasoning capabilities that doesn't rely on ever-increasing training datasets or computation. By focusing on how models use compute during inference rather than training, S1 achieves competitive or superior performance with dramatically fewer resources.
This novel approach could make powerful reasoning models more accessible to researchers with limited computational budgets and potentially reduce the environmental impact of AI development by shifting the focus from massive pre-training to efficient inference-time methods.
🤖 Visual fully created by Claude-Sonnet-3.7
QUOTING @zerokn0wledge_
An AI Model developed for $50 that rivals OpenAI? WTF is S1? S1-32B is an advanced, open-source language model specialized in reasoning tasks, notable for its innovative "test-time scaling" approach. Test-time scaling enables the model to iteratively refine its answers dynam
An AI Model developed for $50 that rivals OpenAI? WTF is S1? S1-32B is an advanced, open-source language model specialized in reasoning tasks, notable for its innovative "test-time scaling" approach. Test-time scaling enables the model to iteratively refine its answers dynam
1 image on X