Large language models (LLMs) are now widely used, but they can be vulnerable to producing harmful content. While safety training and "red teaming" help build in safeguards, new research shows these protections can be overridden. This research explored how fine-tuning open-source LLMs with harmful data can increase the risk of unsafe outputs. Surprisingly, the study found that introducing harmful data during fine-tuning significantly increased the attack success rate (ASR) by 35% compared to the original model. This means a seemingly safe model could be manipulated to generate harmful content. The study also explored the flip side: could fine-tuning with safe data make a model *more* safe? The answer was a resounding yes, with a 51.68% decrease in ASR compared to the baseline. However, there's a catch. Fine-tuning, even for safety, can introduce "knowledge drift," making the model less accurate when faced with false information. This drift was particularly pronounced in models fine-tuned with harmful data, raising concerns about their reliability. The researchers tested this by evaluating how well the models answered trivia questions, sometimes providing false information alongside the question. Harmful models were easily swayed by the false info, while safer models, though slightly impacted, remained more robust. This research highlights the importance of careful fine-tuning and ongoing monitoring for open-source LLMs to ensure responsible and safe use. Future research could explore ways to mitigate harmfulness and further enhance safety measures in these powerful models.