Large language models (LLMs) are trained on massive datasets of text and code. But how does that code influence an LLM's ability to perform tasks *beyond* coding? Researchers explored this in "To Code, or Not To Code? Exploring Impact of Code in Pre-training." They discovered that code plays a crucial role, acting like a hidden booster for an LLM's general abilities. The team ran extensive experiments, training models of various sizes and tweaking the amount and type of code mixed into the training data. The findings were consistent: including code improved performance on tasks like reasoning, general knowledge, and even text generation. One interesting twist: *quality* code matters. Throwing in any old code isn't enough. Models trained with well-written, formally verified code performed significantly better than those trained on less-structured code. This suggests that a little bit of good code goes a long way in shaping a well-rounded LLM. Furthermore, using code in the later "cooldown" phase of training led to additional performance gains. This is the part of training where high-quality data sources get more attention, helping to refine the LLM's abilities. While code clearly helps, there's a balance. Too much code and the model's performance on non-coding tasks can actually decrease. The sweet spot seems to be around 25% code in the initial training mix. This research provides important guidance for developers building the next generation of LLMs. It highlights the importance of code quality and the strategic use of code throughout the training process.