Large language models (LLMs) are impressive, but their ability to reason like humans remains a mystery. Think of it like this: you can memorize facts for a test, but truly understanding a subject means knowing *how* and *why* those facts connect. A new research paper from KAIST and NAVER AI Lab dives deep into this puzzle, analyzing how LLMs use their knowledge to reason through complex questions. Researchers developed a clever approach, creating a graph-like structure where each node represents a question tied to a specific depth of knowledge. Imagine a pyramid: at the base are simple recall questions (What is an activation function?), the middle layer involves applying concepts (How do different activation functions compare?), and the peak represents strategic thinking (Why is one activation function faster than another?). This hierarchical structure lets them test how LLMs navigate from basic facts to intricate reasoning. They built a dataset called DEPTHQA, filled with challenging science and math questions, then tested various LLMs, ranging from 7 billion to 70 billion parameters. One key finding? Smaller models are like students who crammed for the test—they can sometimes answer complex questions but struggle with the underlying basics. This inconsistency, termed "backward discrepancy," highlights a weakness in their true understanding. Larger models fare better but still face a “forward discrepancy,” stumbling when connecting simpler ideas to solve the bigger puzzle. The research suggests that even the most powerful LLMs can struggle with multi-step reasoning. It’s like having all the ingredients for a complex dish but not knowing the recipe. However, when researchers gave the LLMs hints, guiding them through intermediate steps, their performance improved across the board. This discovery points towards new strategies for building LLMs that truly understand, not just memorize.