Fine-tuning large language models (LLMs) to handle specific tasks often requires sensitive data, raising privacy concerns. LLMs can inadvertently memorize and leak this information, posing a significant risk. Differential Privacy (DP) offers a solution by adding noise during training to mask individual data points, but it comes with performance and computational costs, especially when updating all model parameters. A promising alternative lies in Parameter-Efficient Fine-Tuning (PEFT) techniques. Instead of modifying the entire model, PEFT focuses on a smaller subset of parameters, offering both computational efficiency and inherent privacy benefits. This research dives into the privacy implications of PEFT, demonstrating how it reduces leakage compared to traditional fine-tuning, even without DP. The study explores popular PEFT methods like LoRA and Adapters, showing they achieve comparable accuracy while requiring fewer resources. Experiments with intentionally mislabeled data reveal how PEFT methods are less prone to memorization. While Differential Privacy remains a powerful tool, the study interestingly finds that its effectiveness is somewhat diminished when combined with PEFT. This points towards future research in optimizing DP for PEFT architectures, potentially by tailoring noise application for maximum privacy preservation. Overall, this work highlights PEFT as a compelling path towards more private and efficient LLM fine-tuning, especially when full-blown DP is computationally challenging. This opens exciting new possibilities for deploying LLMs in privacy-sensitive applications.