Imagine a world where language barriers are effortlessly broken down, where spoken words seamlessly transform into written text in another language. This isn't science fiction; it's the exciting reality of speech-to-text translation (S2TT). Researchers are constantly exploring new ways to make S2TT more accurate and efficient, especially for multiple languages at once. This post explores an innovative approach using large language models (LLMs), the same technology behind tools like ChatGPT, in a novel way to achieve significant improvements in S2TT. Traditionally, S2TT has been a two-step process: first converting speech to text (like dictation) and then translating that text. This new research, however, directly translates speech into another language, skipping the intermediate step. Using a 'decoder-only' LLM architecture, the model takes encoded speech representations and generates translations in one go, avoiding the pitfalls of traditional methods. This direct approach has shown surprising success on benchmark datasets like CoVoST 2 and FLEURS, achieving top results without relying on massive proprietary datasets. The key innovation lies in feeding continuous speech representations, similar to sound waves, directly into the LLM instead of converting speech into discrete text first. This seemingly small tweak significantly improves translation quality and simplifies the process, making it easier to align speech and text inputs. While fine-tuning LLMs for specific tasks is computationally expensive, researchers have optimized the training process using a technique called LayerNorm and Attention (LNA) fine-tuning. This targets specific parts of the LLM, making the process more efficient and preventing it from ‘forgetting’ its existing knowledge during fine-tuning. This research not only pushes the boundaries of S2TT performance but also sheds light on best practices for LLM training and deployment. It highlights the importance of continuous speech representation and reveals why freezing the speech encoder or using suboptimal methods like LoRA can hinder performance. The future of multilingual communication looks brighter than ever, and LLMs are leading the charge.