Large language models (LLMs) have shown remarkable capabilities in various reasoning tasks, often using a chain-of-thought (CoT) approach to break down complex problems into smaller, manageable steps. But are these LLMs genuinely reasoning step by step, or are they just cleverly disguised guessers? New research suggests that the answer is more nuanced than we might think. The study introduces a novel method called "Chain-of-Probe" (CoP), which acts like a mind-reader, peeking into the LLM's decision-making process at each reasoning step. By examining the model's confidence levels at each stage, researchers discovered a surprising phenomenon called "early answering." In many cases, LLMs arrive at the correct answer almost instantly, before even generating a full chain of thought. This raises a fundamental question: if the model already knows the answer, is the subsequent reasoning process even necessary? The research reveals that early answering is closely linked to the difficulty of the task. For simpler questions, the LLM often jumps to the correct conclusion without much deliberation, suggesting that CoT might be overkill in these scenarios. However, for more complex problems, the CoT becomes crucial, guiding the model towards the right answer through careful step-by-step reasoning. Interestingly, the study also found that a higher confidence level during the reasoning process correlates with a higher likelihood of getting the answer right. This insight led to the development of a "CoP score," which evaluates the quality of the LLM's reasoning process and could be used to prioritize answers with stronger reasoning. While this score helps identify potentially better solutions, it doesn't guarantee flawless reasoning. Further investigation revealed that a significant portion of correct answers were derived from flawed reasoning processes. This has significant implications for the way we evaluate LLM reasoning abilities. Current benchmarks primarily focus on the correctness of the final answer, potentially overlooking errors in the reasoning process. The study underscores the need for more robust evaluation methods that delve deeper into the LLM's "thought" process. The researchers also developed a "CoP Tree," a decision tree that leverages patterns in confidence changes to detect potential errors in reasoning. By identifying and correcting these flaws, they observed substantial improvements in the model’s overall reasoning accuracy. The findings presented in this research open exciting new avenues for understanding how LLMs reason and pave the way for developing more reliable and accurate AI systems. While the CoP method has limitations, it offers a valuable tool for probing the inner workings of LLMs, challenging our assumptions about their reasoning processes, and inspiring future research in this crucial area.