Imagine an AI taking your Italian high school exams. How would it fare against real students? That’s precisely what researchers explored in a new study using the INVALSI tests, Italy’s standardized national assessments, as a benchmark to evaluate the proficiency of Large Language Models (LLMs). These AI systems, trained on massive amounts of text data, have shown impressive language skills, but how do those skills measure up against real-world academic challenges? The study adapted INVALSI tests, covering reading comprehension, grammar, and vocabulary, into a format suitable for automated LLM evaluation. They tested a diverse range of models, from well-known names like OpenAI's GPT models and Google's Gemini to open-source and Italian-specific models. The results? While AI excelled in some areas, particularly reconstructing the meaning of text, it struggled in others. For instance, accurately placing the letter 'h' in Italian words, a seemingly simple task for humans, proved difficult. The study highlighted a common issue: AI models often struggle with tasks that require precise, rule-based understanding of grammar. Interestingly, larger models generally performed better, but even the most advanced couldn’t match average student performance on higher-grade tests. This suggests that while LLMs are powerful tools, they still lag behind humans in complex reasoning and nuanced language understanding, especially in academic contexts. One intriguing aspect of the research is the comparison of human and model performance. The researchers analyzed human accuracy data from several grades and found that while AI tends to perform worse in higher grades, there was no correlation between grade and human performance. The best human scores were recorded in 8th grade. However, this study is just the beginning. Researchers plan to expand the benchmark to include math and multimodal questions, allowing AI to tackle geometry problems and other visual information. The ongoing project will also include a public leaderboard, creating a competitive arena for researchers to improve their LLMs' language proficiency. The study raises interesting questions about the nature of intelligence and learning. While AI may excel in some areas, it still has some way to go before it can truly grasp the nuances of human language and reasoning. The study underscores the fact that while AI is rapidly evolving, there is still a gap between its capabilities and those of human learners, especially in the complex settings of high-stakes academic exams.