Large language models (LLMs) have shown remarkable abilities, but they can stumble when faced with complex, multi-step problems. Their auto-regressive nature—generating text by predicting the next word—often leads to errors and inconsistencies in longer outputs. Why? Because they lack deliberative planning: the ability to carefully consider different paths before committing to a solution. Imagine trying to solve a math problem by simply guessing numbers sequentially—you’re likely to get lost without a plan. That’s where Q* comes in. Researchers have developed this innovative framework to empower LLMs with the ability to strategically ‘think’ through problems. Q* transforms multi-step reasoning into a guided search process, like finding the best route on a map. It learns to estimate the 'value' of each reasoning step by using a proxy Q-value model, helping the LLM select the most promising direction at every turn. This is a game-changer, improving accuracy without the computationally intensive process of fine-tuning entire models. Q* stands out by not needing handcrafted utility functions for each problem type, making it flexible and adaptable across different domains like math problem-solving and code generation. Tests show Q* dramatically improves LLMs' performance, enabling them to solve complex problems with greater accuracy than before. Q* isn’t just a patch-up—it's a significant step towards making LLMs more robust, resourceful, and truly intelligent reasoners.