Imagine training an AI model to translate languages, not with examples painstakingly translated by humans but with high-quality data generated by another AI! That’s the intriguing idea explored in new research from Google, which introduces ‘NewsPaLM,’ a dataset created by a large language model (LLM). Traditionally, machine translation models learn from massive datasets of text scraped from the web. However, this data can be noisy and inconsistent, hindering performance. The NewsPaLM dataset takes a different approach: an LLM generates translations of news articles, refined through techniques like Minimum Bayes Risk (MBR) decoding and Quality Estimation (QE) reranking, resulting in a smaller but much cleaner dataset. Astonishingly, a translation model trained on this LLM-generated dataset outperformed a model trained on a web-crawled dataset 300 times larger! This suggests that high-quality, synthetic data can be remarkably effective for training machine translation models. The research delves deeper, demonstrating the importance of training with multi-sentence examples ('blobs') to improve translation quality for longer texts like paragraphs. Moreover, ‘self-distillation,’ where the LLM that generated the data is further fine-tuned on that same data, yielded even better results, surpassing traditional few-shot learning. While computationally intensive, this approach opens exciting possibilities. The creation of such high-quality datasets presents a significant bottleneck due to the computational cost of the generation and refinement process. However, the remarkable efficiency and performance gains observed with NewsPaLM pave the way for further exploration of LLM-generated data in machine translation and other tasks, potentially revolutionizing how we train and deploy AI models.