AI Medical Assistant
Industry
Healthcare
Client
Vilnius University Hospital Santaros Klinikos
Team
SCALE
Year
2024
We were hired to build two Lithuanian-language LLMs from scratch: one that analyzes business documents and one that transcribes doctor-patient conversations with medical-grade accuracy. This is because off-the-shelf models don't exist for a language spoken by three million people.
Challenge
Lithuanian is a low-resource language. There are no production-ready large language models that handle Lithuanian medical terminology, the kind of vocabulary where precision isn't optional and a misheard word can change a diagnosis. Physicians at Vilnius University Hospital Santaros Klinikos (VULSK) were doing what doctors everywhere do when automation isn't available: spending hours on documentation that could be captured from conversations they're already having.
The business case ran parallel. Lithuanian organizations handling contracts, invoices, and cross-platform communications had no AI tools that could scan, interpret, and extract insights from their documents without manual processing. Diverse formats, strict data security requirements, and the absence of Lithuanian-language training data meant no existing solution could be dropped in.
Building for Lithuanian meant starting from zero on training data. The project required physicians at VULSK to collect real doctor-patient audio recordings like domain-specific data that no public dataset could substitute and the models needed to handle the gap between everyday Lithuanian and clinical vocabulary, where terms differ significantly and context determines meaning.
Approach
The project produces two LLMs that share a Lithuanian language foundation but serve different domains.
The business documentation model scans, analyzes, and summarizes contracts, invoices, communications and generates financial performance insights from document content. All processing stays within company infrastructure, which was a non-negotiable security requirement. This is the more conventional of the two models, but building it for Lithuanian still required addressing the low-resource language gap that makes standard fine-tuning approaches insufficient.
The medical speech recognition model is the more ambitious build. It listens to doctor-patient conversations, transcribes them with medical-terminology accuracy, and produces structured clinical summaries. The goal is to automate the documentation workflow that currently pulls physicians away from patient care. It captures what's said in the room and turnes it into usable records without manual transcription.
The training data for the medical model comes directly from VULSK, where physicians are recording real clinical conversations. This is the only way to teach the model the specific vocabulary, phrasing, and contextual nuances of medical Lithuanian. The models were built on LLaMA architectures (7b through 70b parameters) paired with Wav2vec 2.0, Whisper, and AssemblyAI for speech recognition, with extensive testing focused on maintaining accuracy under the imperfect audio conditions typical of clinical settings.
Architecture/Backend
Document analysis & clinical speech-to-summary pipelines; secure on-premise processing.
AI/ML models
LLaMA tuned for Lithuanian; Wav2vec/Whisper/AssemblyAI; trained on VULSK clinical audio.
Infrastructure/Deployment
Live VULSK data collection; phased training; full clinical deployment by mid-2026.
Key results
Developing Lithuanian-language LLMs where none existed, a low-resource language build that establishes a replicable framework for other underserved languages facing the same gap.
Business model targeting 50% reduction in document analysis time with over 90% categorization accuracy, automating extraction and insight generation from Lithuanian-language business documents.
Medical model projected to halve clinical documentation time at up to 95% transcription accuracy for medical terminology, converting physician hours from paperwork back to patient care.
Built a clinical data pipeline at VULSK where physicians record real consultations for model training, producing the domain-specific Lithuanian medical dataset that no public corpus could provide.
