Fine-tuning an LLM to generate clinical notes from audio transcriptions

Abstract

Clinical documentation is a critical but time-consuming burden for healthcare professionals, often contributing to burnout and reducing time for patient care. While Large Language Models (LLMs) offer a promising solution to automate the generation of clinical notes from patient-doctor dialogues, their adoption in Europe faces two major hurdles: strict privacy regulations (GDPR) that discourage the use of external cloud-based APIs, and a scarcity of models optimized for European Portuguese (PT-PT). This thesis addresses these challenges by developing and evaluating a privacy-preserving pipeline for automated clinical note generation in PT-PT. We investigate the viability of adapting and self-hosting open-weight LLMs through Parameter-Efficient Fine-Tuning, specifically QLoRA, with the main long-context benchmark covering Qwen3 8B, Llama 3.1 8B, and Gervásio 8B, and a secondary short-context benchmark covering BioMistral 7B, Llama 3.1 8B, and EuroLLM 9B. The study uses an anonymized dataset of real-world telemedicine transcripts to train models to produce structured notes following the specific Knokcare format (analogous to SOAP). A central contribution of this work is to demonstrate the operational viability of generation from anonymized inputs. However, the exact magnitude of the privacy-utility trade-off remained open because the thesis did not include a matched raw-versus-anonymized ablation. Furthermore, we propose a robust evaluation framework that moves beyond traditional metrics, employing a dual-vector "LLM-as-a-judge" approach to detect hallucinations and assess critical omissions, documentation utility, and transcript-grounded factual consistency. The results show that fine-tuning consistently improves structure, documentation utility, and PT-PT clinical style. However, proprietary models retained an advantage in safety and critical-omission control, so the most defensible use case is clinician-in-the-loop drafting assistance with mandatory clinical review rather than autonomous clinical note writing.