Large language models (LLMs) are increasingly used for complex reasoning tasks, from solving math problems to generating code. However, the algorithms that power this reasoning often require significant compute resources, leading to longer response times and higher costs. Imagine an LLM trying to solve a complex equation. It might explore multiple solution paths, much like a human, but each path requires processing power and time. This can quickly become a bottleneck, especially when serving multiple users simultaneously. Existing LLM serving systems struggle to efficiently allocate resources to these complex reasoning tasks, often over-allocating resources to simple queries while under-allocating to harder ones. This inefficiency wastes valuable compute power and increases latency. A new system called Dynasor addresses this challenge by intelligently managing how these reasoning programs use compute resources. Dynasor introduces the concept of “certaindex,” a measure of how confident the LLM is in its current solution path. Think of it as the LLM's way of saying, “I’m pretty sure I’m on the right track.” Dynasor leverages certaindex to dynamically adjust resource allocation. If the LLM expresses high certainty, Dynasor reduces the allocated resources, preventing wasted compute on problems that are nearly solved. Conversely, if the LLM is uncertain, Dynasor allocates more resources, allowing the LLM to explore more solution paths. This dynamic approach not only saves compute but also improves response times. Dynasor also uses a technique called “gang scheduling” to further enhance efficiency. By grouping related requests together, Dynasor minimizes context switching and maximizes resource utilization. This means fewer delays and quicker responses for users. Experiments across various LLM reasoning tasks, including math problem-solving and code generation, show that Dynasor significantly outperforms existing systems. It achieves the same accuracy with considerably fewer compute resources, reducing costs and improving response times. In some cases, Dynasor reduced compute usage by up to 50% while maintaining accuracy, and in online settings, it handled up to 3.3 times more queries than comparable systems. Dynasor’s innovative use of certaindex and gang scheduling opens new possibilities for efficiently serving LLM-based reasoning applications. As LLMs continue to evolve and tackle more complex reasoning challenges, systems like Dynasor will play a crucial role in making these powerful models more accessible and efficient.