Large Language Models (LLMs) are rapidly evolving, demonstrating impressive abilities across various tasks. But how well can they truly reason and strategize? A new benchmark called GAMEBOT puts LLMs to the test, challenging them not just to win games, but to reveal the *how* and *why* behind their decisions. Researchers have designed a unique gaming arena where LLMs face off in eight different games, ranging from classic board games like Othello and Checkers to more complex scenarios like Texas Hold'em and negotiation simulations. What sets GAMEBOT apart is its focus on transparency. Instead of simply evaluating wins and losses, GAMEBOT delves into the LLM's thought processes by breaking down complex decisions into smaller, modular sub-problems. Using carefully crafted prompts, researchers guide the LLMs to explain their reasoning at each step, providing a glimpse into their strategic thinking. The results reveal that while some LLMs, like GPT-4 and Claude 3.5, show promising strategic abilities, even the most advanced models struggle with certain aspects of reasoning. For example, an LLM might accurately predict the trajectory of a ball in Pong but fail to adapt its strategy beyond simply centering its paddle. This highlights the importance of understanding *how* LLMs arrive at their decisions, not just whether they succeed or fail. GAMEBOT offers a valuable tool for evaluating and improving LLM reasoning, paving the way for more strategic and transparent AI systems in the future. The benchmark also revealed unexpected inconsistencies. LLMs that excelled in one game sometimes faltered in another, suggesting that transferring knowledge and adapting to new rules remains a challenge. The relatively low scores across the board indicate that complex reasoning in games is a difficult hurdle for LLMs to overcome. This research underscores the importance of moving beyond simple outcome metrics and delving into the intricacies of LLM decision-making. By understanding where LLMs stumble, we can better refine their training and unlock their full potential for strategic thinking in the real world.