-
Steering for Truth, and for Lies
A short empirical post on using an Apollo-style honesty/deception direction to steer Qwen models toward disclosure or deception, with roleplay, sandbagging, and prompt-interaction results.
-
GRPO, SFT, and teaching reasoning through arithmetic
What GRPO and SFT can and cannot teach a 3B model about arithmetic reasoning, measured on the Countdown task.
-
Autonomous math agent on Erdős problems
A pipeline to triage and attack open Erdős problems at scale with the Rethlas agent, and a look at the five results we got.
-
Improving one small model: a deep look at depth-recurrence in 10-minute pretraining
We take one depth-recurrent language model from a 10-minute pretraining competition and try three ways to improve it. Two fail cleanly; the third, learning a mixing rule and then freezing it, ships.
-
Parameter Golf: Six Weeks to Build the Best LLM
An account of OpenAI's Parameter Golf competition. Six weeks, two thousand pull requests, and a 14% compression improvement wrung from the same hardware, the same data, and the same ten minutes of training.