Large language models (LLMs) excel at many tasks, but can they truly reason strategically? New research puts LLMs to the test using classic games from behavioral economics like the p-Beauty Contest, the Guessing Game, and the Money Request Game. These games require players to anticipate and adapt to the actions of others, revealing the depth of their strategic thinking. The results are surprising. While most LLMs struggle with higher-order strategic thinking, often exhibiting behavior comparable to random guessing, OpenAI's GPT-o1 consistently demonstrates superior performance, even surpassing human players in several scenarios. This discrepancy highlights the critical role of reinforcement learning and chain-of-thought prompting in developing advanced reasoning capabilities in LLMs. GPT-o1, trained using these techniques, excels by breaking down complex tasks into smaller, manageable steps, allowing it to learn and adapt more effectively. The research reveals the significant limitations of current LLMs in navigating multi-agent environments and emphasizes the importance of refining training methods to enhance their strategic reasoning skills. This is particularly crucial as LLMs are increasingly used in real-world applications requiring complex interactions and strategic decision-making, such as planning, education, healthcare, and autonomous agents. The future of LLMs hinges on their ability to not just understand language, but to truly reason and strategize like humans—a challenge that GPT-o1’s success suggests is within reach.