Imagine teaching a brilliant but generalist AI to become a specialized expert. It's like training someone who knows a lot about everything to become a doctor, lawyer, or mathematician. The traditional approach has been to immerse the AI in a sea of domain-specific data, like medical textbooks, hoping it absorbs the knowledge. This process, continual pre-training, helps the AI memorize facts but often struggles to connect them effectively, similar to how rote memorization without understanding context can be ineffective for humans. The problem becomes even more complex when you consider formatting. It's not enough for the AI to have medical knowledge; it also needs to communicate that knowledge in a doctor's style, using the correct terminology and phrasing. Existing methods often try to teach knowledge and format simultaneously, resulting in a confusing learning process for the AI. Researchers propose a novel two-stage framework called Mix-CPT that separates, or decouples, knowledge learning from format alignment. First, Mix-CPT uses a "knowledge mixture" approach, blending raw domain data (like medical texts) with general instructions and conversational examples. This gives the AI a taste of how the knowledge is used in practice, improving its understanding and memorization. To prevent the AI from forgetting its general knowledge during this specialized training (a common problem called catastrophic forgetting), the researchers introduced a technique called Logit Swap Self-Distillation. This acts like a memory aid, gently nudging the AI to recall its existing knowledge base while learning new information. Once the AI has a good grasp of the domain knowledge, Mix-CPT moves to the second stage: format alignment. Here, the AI is fine-tuned using a small set of carefully selected examples that match the desired output style (like a doctor's bedside manner). By decoupling these two learning stages, Mix-CPT allows the AI to focus on one task at a time, making the process more efficient and effective. Experiments on various domains, from encyclopedic knowledge and math to coding, show that Mix-CPT significantly improves the AI's ability to perform specialized tasks while retaining its general intelligence. This breakthrough paves the way for training specialized AIs in fields like medicine, law, and scientific research, bringing us closer to realizing the full potential of artificial intelligence in solving complex real-world problems.