Large Language Models (LLMs) have wowed us with their ability to perform complex reasoning tasks, often using a chain-of-thought (CoT) approach to break down problems into smaller steps. But how can we truly understand what’s happening inside these AI giants as they reason? Current methods for evaluating CoT either rely on expensive, manually annotated data or fall short in accurately assessing the individual reasoning steps, sometimes leading to misleading conclusions. This research explores a new way to peek into the ‘black box’ of LLM reasoning using information theory. Imagine being able to measure the 'information gain' at each step of an LLM's thought process. This research proposes a framework to do just that, quantifying how much each step contributes to arriving at the correct final answer. This information gain metric acts as a magnifying glass, allowing us to identify where the LLM’s reasoning might be going astray without the need for labeled CoT data. The researchers tested their framework on various datasets, including a simplified mathematical reasoning task and the more complex GSM-8K dataset. The results are encouraging. This new information-theoretic approach consistently outperformed existing methods, providing a more accurate and nuanced view of how LLMs reason. This approach revealed some interesting insights. For example, in one experiment, an LLM struggled to add large and small numbers together, a subtle error that other methods missed. This granular level of analysis is crucial for understanding the strengths and weaknesses of LLMs. While this research presents a significant step forward, challenges remain. The current method requires training a separate 'supervisor' model, which can be computationally expensive. Future work might explore streamlining this process through in-context learning. Despite these challenges, this innovative approach offers a promising new avenue for understanding and improving the reasoning abilities of our increasingly sophisticated AI systems.