Large language models (LLMs) have shown remarkable capabilities in various tasks, including translation. However, most research has focused on translating individual sentences. Could LLMs be secretly better at translating entire documents than we give them credit for? New research challenges the conventional wisdom that LLMs need specialized training to handle document-level translation effectively. It turns out that simply prompting an LLM to translate a whole document at once, instead of sentence by sentence, can lead to surprisingly good results—even without any specific document-level training. The catch? Our standard evaluation metrics, like the commonly used BLEU score, might be giving us a skewed picture. Traditional metrics often favor the choppy output of sentence-by-sentence translation because they primarily focus on word-for-word accuracy. They miss the bigger picture – the overall flow and coherence of the translated document. Think about it: a document isn't just a collection of individual sentences; it’s a cohesive narrative. To get a more accurate view, researchers turned to a novel evaluation method: using another LLM, like GPT-4, as the judge. They prompted GPT-4 to assess the translated documents based on fluency, accuracy, and cohesion—how well the sentences flow together logically and grammatically. The results were revealing. When evaluated by GPT-4, document-level translations often outperformed sentence-by-sentence translations in fluency and overall coherence. This suggests that LLMs have an inherent ability to grasp the context of an entire document, leading to more natural and meaningful translations. This research has significant implications for how we evaluate and utilize LLMs for translation. It highlights the limitations of relying solely on traditional metrics and points towards the potential of LLMs to revolutionize how we translate longer, more complex texts. The future of machine translation might be less about specialized training and more about unlocking the hidden potential already within these powerful models.