Imagine a robot bartender flawlessly mixing cocktails, a construction crew effortlessly assembling complex structures, or a mechanic expertly changing a tire. These scenarios, seemingly mundane for humans, pose significant challenges for artificial intelligence. Recent Large Language Models (LLMs) have shown impressive abilities in language tasks, but how do they fare when tasked with actual planning? A new study dives deep into the planning abilities of OpenAI's o1 models, exploring not just whether they can devise a plan, but how efficient and adaptable those plans are. The research focuses on three key aspects: feasibility (can AI create a workable plan?), optimality (can it find the most efficient solution?), and generalizability (can the same AI tackle different planning scenarios?). The researchers put o1 to the test across various simulated tasks, like bartending, block stacking, and even tire changes. While o1 showed promise in following pre-defined rules in simpler tasks, complex scenarios requiring spatial reasoning and multi-step actions exposed its limitations. For instance, the AI excelled at following instructions for assembling blocks in a specific order, demonstrating impressive constraint-following abilities. However, in a more complex construction task involving 3D structures, o1 faltered, often misinterpreting spatial relationships or skipping crucial steps. This suggests that while o1 can manage simple, sequential tasks, it struggles when faced with tasks requiring a deeper understanding of the environment and how actions impact the overall state. The gap between feasibility and optimality was another key finding. Even when o1 managed to devise a working plan, it often wasn't the most efficient. It sometimes added unnecessary steps or failed to optimize resource usage, highlighting a crucial area for improvement. Generalizability also proved to be a hurdle. While o1 performed admirably in some new scenarios, it struggled when the context became too abstract or when symbolic representations replaced familiar language cues. For example, o1 could efficiently plan tire changes with clear instructions but stumbled when those instructions were replaced with random symbols, even though the underlying logic remained the same. This research underscores the importance of looking beyond simple success rates when evaluating AI planning. By examining feasibility, optimality, and generalizability, the study unveils a more nuanced understanding of o1's capabilities and reveals critical areas for future research. As LLMs evolve, addressing these limitations will be essential for creating truly intelligent planning agents capable of handling real-world complexities, from coordinating robot actions to optimizing logistical operations.