Large language models (LLMs) have surprised us with their ability to perform tasks requiring complex reasoning, including solving math problems. But how do they actually do it? Are they learning genuine mathematical algorithms, or is something else going on? New research suggests a surprising answer: LLMs aren't crunching numbers like a calculator; instead, they use a clever collection of heuristics—simple rules and pattern-matching tricks—to arrive at the right answer. Researchers dove deep into the inner workings of several LLMs, including Llama 3 and Pythia, to uncover this “bag of heuristics” approach. By analyzing the activation patterns of individual neurons within the models, they discovered that specific neurons fire in response to particular numerical patterns in the input. For instance, one neuron might activate strongly when both numbers in a subtraction problem are even. These neurons, in turn, boost the probability of related output tokens, effectively guessing the correct answer based on the input patterns. Think of it like a seasoned chef who can estimate ingredient quantities without precise measurement, relying on experience and rules of thumb. This “bag of heuristics” strategy works remarkably well for many math problems, but it also reveals why LLMs sometimes stumble. They're not employing a universal algorithm, so their performance can be inconsistent, especially when faced with problems outside the patterns they’ve memorized. This research has important implications for the future of AI. It suggests that we might need to rethink how we train LLMs if we want them to achieve true mathematical reasoning. Simply scaling up model size might not be enough; we need to encourage them to develop more robust, generalizable strategies for problem-solving, rather than relying on a patchwork of memorized tricks. The discovery of this heuristic-based approach opens up exciting new avenues for research into LLM interpretability and highlights the fascinating and sometimes unexpected ways these complex models navigate the world of numbers.