Large language models (LLMs) are like eager students—they quickly learn, but sometimes at a cost. When fine-tuned on instructions, LLMs exhibit a peculiar behavior: they get better at following directions, but their general knowledge and reasoning abilities can suffer. This is known as the "alignment tax." New research explores this phenomenon, suggesting that LLMs, in their quest to please, overfit to biases in the instruction data. Think of it like studying only the practice tests—you might ace the test, but forget the underlying concepts. The paper introduces a clever "disperse-then-merge" strategy. Instead of training one LLM on the entire dataset, they train multiple smaller models on different portions of the data. Each model learns unique biases, but when merged together, these biases cancel each other out, resulting in a model that retains general knowledge while excelling at following instructions. This approach outperforms other methods like data filtering and regularization, offering a promising way to train more robust and knowledgeable LLMs. The implications are significant. Imagine LLMs that can write code, answer complex questions, and follow instructions without sacrificing their understanding of the world. This research is a step towards that future, paving the way for more capable and trustworthy AI assistants.