The world of large language models (LLMs) is expanding rapidly, but not all languages are benefiting equally. Many LLMs show a significant performance gap between high-resource languages like English and low-resource languages. Researchers at Tianjin University are working to bridge this gap with their new open-source multilingual LLM, FuxiTranyu. This 8-billion parameter model is trained on a carefully balanced dataset of 600 billion tokens, spanning 43 natural languages and 16 programming languages. The team’s focus on balanced data is key. Previous multilingual LLMs often prioritize high-resource languages, leading to inconsistent performance across different languages. FuxiTranyu aims to solve this by giving equal weight to all languages during training. They’ve also released two instruction-tuned versions of the model: FuxiTranyu-8B-SFT, fine-tuned on various instruction datasets, and FuxiTranyu-8B-DPO, further refined with Direct Preference Optimization (DPO) for better alignment with human instructions. Benchmarks show FuxiTranyu’s competitive performance against existing multilingual LLMs like BLOOM and PolyLM. Interestingly, analysis of the model’s internal workings shows that FuxiTranyu learns more language-agnostic representations compared to models like BLOOM, which is likely due to the balanced training data. While languages with very limited resources still face challenges, the developers are making strides. FuxiTranyu is open-source, meaning it’s available for anyone to use, modify, and build upon. This open access is crucial for accelerating research and development in multilingual LLMs. The release of FuxiTranyu, including its training checkpoints, provides a valuable resource for researchers exploring multilingual natural language processing, paving the way for a more inclusive future for LLMs where all languages have a voice.