Large language models (LLMs) have revolutionized machine translation, but how do they actually use the context they're given? New research dives deep into the inner workings of LLM-based translation, revealing surprising insights into how these models leverage examples and source text. The study, using LLaMA-2 and the specialized translation model TOWER, uncovers a fascinating positional bias: the earlier an example appears in the context, the more influential it is. Imagine giving an LLM several translation examples before asking it to translate a new sentence. This research shows that the model pays more attention to the initial examples than the later ones. This 'positional bias' is particularly pronounced in LLMs not specifically trained on parallel translation data. Adding a task description like "Translate from English to German" doesn't override this bias, suggesting it’s not simply about the order of words but something more fundamental about how LLMs process information. However, this bias can be disrupted. When the researchers included a copy of the very sentence the LLM was supposed to translate, along with its correct translation, as the last example, the model often caught on and copied the answer, diminishing the positional bias. The research also reveals that the source text of an example is generally more important than the target text. This mirrors findings in traditional machine translation models and suggests a consistent focus on understanding the original language. Perhaps the most intriguing finding relates to translation errors, specifically 'hallucinations' where the LLM fabricates information. The study found that low contributions from the source text can often predict these errors, particularly in the TOWER model. In such cases, the model might be overly reliant on examples and less on the actual sentence it's supposed to translate, leading to inaccuracies. This research opens a window into the complex world of LLM translation. By understanding how these models use context, we can develop strategies to improve their accuracy and reliability, moving us closer to truly seamless cross-lingual communication. Future research could explore how these findings hold up with even larger models and different LLM architectures.