Large Language Models (LLMs) hold immense promise for healthcare, but using them with sensitive Electronic Health Records (EHR) presents challenges due to privacy concerns and computational limitations. A new compact LLM framework tackles these issues head-on. This innovative approach uses preprocessing techniques, like regular expressions, to filter and highlight crucial information in clinical notes, boosting the performance of smaller, locally-deployed LLMs. This allows researchers and clinicians to leverage the power of LLMs while keeping sensitive data secure and within the reach of standard computational resources. Tests on both private and public EHR datasets show that this preprocessing approach significantly improves the accuracy of smaller LLMs, especially in zero-shot learning scenarios. This opens doors for more efficient and privacy-preserving applications of LLMs in healthcare, enabling crucial tasks like disease phenotyping without the need for massive computational power or compromising patient privacy. While more sophisticated NLP techniques could be incorporated in the future, this framework provides a valuable starting point for optimizing LLM performance in sensitive, data-intensive tasks while addressing real-world computational and privacy limitations.