Large language models (LLMs) have shown impressive abilities, even solving complex problems. But how deeply do they understand what they're doing, especially with something like grade-school math? A new research paper, "Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process," dives into the inner workings of LLMs tackling math. Rather than just focusing on how *well* LLMs perform on math benchmarks, the researchers designed a system to create tons of unique grade-school math problems—preventing the LLM from simply memorizing answers. They discovered some fascinating things. First, LLMs can indeed learn reasoning skills, generalizing to longer problems than they were trained on. Even more intriguing, the models often find the *shortest* solution, suggesting they're not just blindly crunching numbers, but actually planning their approach—a skill not explicitly taught. By probing the model's internal states, the researchers discovered that LLMs solve problems much like humans, mentally gathering the necessary information before starting calculations. Surprisingly, they also learn things *beyond* what humans typically do, like mapping out all relationships between parameters, even if some aren't needed for the immediate problem. This suggests that LLMs might develop 'hidden skills' not explicitly present in their training data. The research also offers insights into why LLMs sometimes make mistakes, often due to systematic errors in their 'planning' phase. Finally, and perhaps unexpectedly, the *depth* of the model (number of layers) appears more important for reasoning than sheer size, challenging some common beliefs about AI scaling. While the study focuses on a simplified version of math, it provides a compelling look into how LLMs think and learn, hinting at a more nuanced relationship between model architecture, training data, and emergent skills.