Imagine a world where game developers could predict difficulty levels without endless hours of human playtesting. That’s the tantalizing prospect offered by using Large Language Models (LLMs) as virtual testers. A fascinating new study explores how LLMs can be employed to measure game difficulty, tackling two popular games as test cases: the word puzzle Wordle, and the deck-building roguelike Slay the Spire. The results reveal a surprising twist: while LLMs might not be champion-level players, their performance correlates strongly with human-perceived difficulty. In Wordle, the more guesses an LLM needed, the harder humans found the puzzle. This held true even when the LLMs weren’t as efficient as human solvers. In Slay the Spire, similar results emerged – LLMs struggled against bosses that also tripped up human players. This suggests LLMs could be invaluable tools for developers. Imagine using an LLM to fine-tune the difficulty curve of a game, ensuring a smooth progression and optimal player engagement. The research provides best practices for effectively utilizing LLMs for playtesting, paving the way for more efficient and insightful game development. However, challenges remain. LLMs still don’t fully grasp the nuances of human play, especially in complex games. Future research aims to address this, exploring how LLMs can learn and adapt, mimicking the skill development of human players. This research opens exciting possibilities. Perhaps one day, game design will involve a back-and-forth between human creativity and AI-driven insight, leading to more engaging and rewarding gaming experiences for all.