Large language models (LLMs) are revolutionizing how we interact with technology, but they also come with quirks. One such oddity, explored in a recent research paper from Google, is the tendency of LLMs to be overly verbose when performing tasks like translation. Instead of just providing the translated text, some LLMs refuse to translate, offer multiple translations, or add lengthy explanations for their choices. This 'chatty' AI behavior presents significant challenges for traditional evaluation methods. The researchers investigated eight different LLMs across multiple language pairs and discovered some interesting patterns. LLMs like Gemini 1.5 Pro were found to be particularly verbose, often providing context-sensitive translations or additional commentary, especially when dealing with short input segments. Others, like GPT-4 and Aya23, remained concise. But why do some LLMs refuse to translate at all? The study points to three main triggers: suspected harmful or copyrighted content and the presence of non-natural language elements like URLs or emojis. The issue isn't just theoretical; it has real-world implications. Current automatic and human evaluation metrics struggle to handle these verbose outputs. Verbose LLMs are penalized, leading to inaccurate performance rankings. Imagine a translation app that refuses to translate a harmless phrase simply because it's too short or contains an emoji! Addressing this verbosity issue is crucial for accurate evaluation. The researchers propose two solutions: either modify LLMs to suppress their chattiness or adapt evaluation methods to account for these nuances. While suppressing verbosity through prompt engineering seems promising, it ignores the potential value of context-sensitive translations. Developing context-aware evaluation metrics presents a greater challenge but could lead to a more meaningful assessment of LLM capabilities. As LLMs continue to evolve, tackling verbosity will be essential for creating truly helpful and reliable AI translation tools.