Large language models (LLMs) have become very good at a wide array of tasks. But can these huge models truly become great translators? In the world of machine translation (MT), simply generating fluent-sounding translations doesn't cut it. How well an LLM actually "understands" what it's translating and captures the nuances of human language comes down to the quality of its output. Researchers have been exploring various techniques to enhance LLM-based translations by directly training these models to align with human preferences. One method that has gained traction is preference alignment, which learns from preferences induced by quality estimators between candidate translations. In this method, an LLM is presented with several possible translations and a "score" for each possible translation, allowing the model to learn to select the translation with the higher score. A new study examines whether this "preference-based alignment" always results in improved LLM translations. In particular, researchers explored Contrastive Preference Optimization (CPO), a cutting-edge approach that seeks to refine the translation quality of LLMs by aligning their outputs with human preferences. They conducted thorough tests on a variety of language pairs to understand how factors like preference data and training approaches affect the quality of translation outputs. Their results were surprising. While CPO consistently outperformed standard Supervised Fine-Tuning (SFT) when given high-quality preference data, the performance gains were highly dependent on how the preference data was generated. The study found that using external high-quality translation systems to generate the candidate translations sometimes helped the LLM learn from its mistakes more effectively, but it could also introduce inconsistencies in translation quality across different languages. The researchers discovered something even more unexpected. Simply using an LLM's own generated candidate translations to fine-tune its performance often resulted in translations that were just as good, sometimes even better, than the models trained on external data. This discovery points to a promising path: LLMs might be capable of self-improvement by simply learning from their own output and fine-tuning to improve their translation quality. These findings have some important real-world implications. First, they highlight the need for careful consideration of the training data and learning objectives in preference-based alignment. Second, they suggest that LLMs have a built-in capacity for self-improvement, which could significantly impact the development of high-quality, LLM-based translation tools. The study concludes that while techniques like CPO hold great potential, fine-tuning LLMs on their own output might be a more straightforward and effective approach for improving translations. It’s a surprising twist in the quest for creating truly human-like translators!