Generating patient letters, a time-consuming task for physicians, is ripe for automation. A new study explores how locally fine-tuned Large Language Models (LLMs), specifically the LLaMA models, can automate physician letter generation while preserving patient privacy within the radiation oncology field. The research reveals that off-the-shelf LLMs struggle to generate useful medical letters without specialized training. However, using a technique called QLoRA, researchers successfully fine-tuned these models on a single hospital workstation without needing massive computing power. This method allows hospitals to leverage their own data to train LLMs locally, addressing privacy concerns associated with sharing sensitive patient information with external AI providers. The fine-tuned LLM learned the specific language and style of radiation oncology and generated letters that closely matched the hospital’s standard format. Interestingly, the smaller 8B LLaMA-3 model outperformed the larger 13B LLaMA-2 model in generating accurate summaries. Physician evaluations highlighted that while the AI-generated letters excelled at standard elements like salutations and treatment histories, they sometimes missed details not explicitly provided in the input data. However, with quick physician review and correction, the automated system offers significant practical value. This research points to a future where AI can handle routine documentation tasks, freeing up clinicians to focus on patient care. However, challenges regarding data privacy and the limitations of information included in the fine-tuned models point to further improvements in utilizing LLMs. Safe and effective integration of AI assistants promises greater efficiency and more time for direct patient interaction, marking a key step towards modernizing clinical workflows.