Predicting international events is a complex puzzle. Traditionally, experts have relied on their understanding of history, politics, and global dynamics, but could AI offer a powerful new approach? Researchers have introduced MIRAI, a novel benchmark designed to test whether large language models (LLMs) can accurately forecast international events. MIRAI isn't your average AI test. It simulates a real-world environment, providing LLMs with access to a massive database of historical events and news articles. Think of it as giving an LLM the tools a human expert would use – except this expert can process information at lightning speed. The LLMs are then challenged to predict future relations between countries, drawing on both structured data (like event records) and unstructured data (like news text). This is where things get interesting. The research team, based at UCLA and Caltech, has designed MIRAI to test short-term and long-term forecasting. They want to see if LLMs can accurately predict events just a few days out, as well as events months down the line. They also want to see how effectively LLMs can use provided software tools via Python code. The initial results are promising, yet highlight the challenges ahead. While the LLMs showed some ability to forecast, the task proved difficult, especially when it came to predicting very specific types of interactions or events far in the future. One key finding was the importance of providing LLMs with the right tools. Those with access to both news and event data significantly outperformed those that only had one type of information. It's like a detective needing both witness testimonies and forensic evidence to solve a case. Another interesting discovery was that more powerful LLMs benefited from being able to write flexible blocks of code, while less powerful ones struggled with this additional complexity. It's a case of "with great power comes great responsibility" – and better coding skills. The research also revealed that getting LLMs to make consistent predictions over multiple attempts was an effective way to boost their accuracy. The MIRAI benchmark is a significant step towards understanding the potential of AI for event forecasting. While there's still a lot of work to be done, the results suggest that LLMs, armed with the right data and tools, could one day become valuable partners in navigating the complex landscape of international relations. Future versions of MIRAI will include richer data sources and tools. This could help reveal even more about how LLMs learn, reason, and predict—not to mention potentially offering a glimpse into the future of geopolitics.