Reconciling fragmented medical records with multimodal AI, structured extraction, and clinical safeguards.
An end-to-end AI platform engineered to extract, reconcile, and organize medication data from handwritten prescriptions and discharge summaries — reducing preventable drug discrepancies.
Can multimodal LLMs safely extract and reconcile complex prescriptions without hallucinating ambiguous handwriting?
Hospital transitions and multi-clinic visits create dangerous information silos. MedScope AI uses Gemini multimodal vision combined with strict JSON schema constraints and deterministic reconciliation logic to detect therapeutic duplicates and dosage shifts before medication errors occur.
Where Medication Records Fragment
When patients move across care transitions—such as from a Primary Care Physician to an inpatient hospital stay, and finally to home discharge—their medication history is split across handwritten scripts, paper discharge summaries, and separate EHR records.
A patient prescribed an ACE inhibitor (e.g. Lisinopril 10mg) prior to admission is often concurrently discharged on an ARB (e.g. Losartan 50mg) by a different team, creating severe dual RAAS blockade risks.
Cursive handwriting, missing strengths, and abbreviated frequencies ("1 tab OD pc") cause confusion for patients and increase preventable readmissions.
How MedScope AI Processes Records
MedScope AI employs a 5-step deterministic ingestion architecture that separates multimodal vision parsing from downstream clinical discrepancy detection:
Multi-Doc Image Ingestion
User uploads prescription images, discharge summaries, or clinical MAR records in PNG, JPG, or WEBP formats.
Multimodal Gemini API Call
Transmits high-fidelity image buffers server-side alongside carefully engineered prompts with temperature calibrated to 0.1.
Deterministic JSON Schema Extraction
Extracts structured medication fields: medicationName, strength, dose, frequency, route, duration, and instructions.
Confidence & Discrepancy Scoring
Automatically flags low-confidence (<80%) or ambiguous handwriting as 'NEEDS VERIFICATION' rather than guessing.
Cross-Record Reconciliation
Cross-references multiple documents (e.g. Prior PCP Rx vs Inpatient Discharge Summary) to detect therapeutic duplicates (ACE + ARB).
From Fragmented Paper to a Verified Medication Record
A 4-phase clinical workflow that ensures continuous human-in-the-loop verification, discrepancy detection, and full audit provenance before EHR commitment:
Non-Hallucination & Security Controls
Explicit negative prompt instructions prevent the model from guessing cursive or blurred handwriting. Items below 80% confidence are marked NEEDS VERIFICATION.
API keys are managed strictly server-side through process.env.GEMINI_API_KEY, eliminating browser token exposure.
Gemini responses are structured to a rigid schema and sanitized through JSON boundary parsers, gracefully handling markdown code fences.
Integrates patient allergy profiles and chronic conditions to flag contraindications without attempting autonomous medical prescribing.
Key Engineering Insights
Building MedScope AI demonstrated that in high-stakes domains like healthcare, the greatest value of Generative AI lies not in raw generation, but in structured extraction, transparency of uncertainty, and enabling human clinicians to make faster, safer decisions.
- Multi-language translation for regional Indian languages (Hindi, Telugu, Tamil).
- Real-time FHIR / HL7 clinical document standard exports.
- Offline PDF daily dosing calendar generation for patient home adherence.