Large Language Models (LLMs) have shown impressive abilities in many languages, but adapting them to low-resource languages like Marathi presents unique challenges. Researchers recently explored these challenges using a technique called LoRA PEFT, which allows for efficient fine-tuning of LLMs without massive computational resources. They used a translated version of the Alpaca dataset, containing thousands of instruction-response pairs, to train several Gemma models. Interestingly, while standard automated metrics suggested a performance decrease after fine-tuning, manual evaluations revealed something quite different. The fine-tuned models often produced more contextually appropriate responses in Marathi, demonstrating the limitations of current evaluation methods for low-resource languages. These automated metrics, focusing on things like sentiment analysis and reasoning tasks, might miss the subtle cultural nuances and linguistic patterns crucial for true understanding. The research highlights the need for better evaluation strategies, possibly incorporating human judgment more heavily, and the critical importance of high-quality, native language datasets for accurately assessing LLM performance in these under-resourced linguistic landscapes. The quest to make AI truly multilingual faces the hurdle of effectively teaching it the subtleties of languages with less digital presence, and this research sheds light on the path forward.