Privacy-First AI for Document Digitization
An on-premise AI system automating the digitization of physical government records containing mixed English and Devanagari scripts, built for strict data sovereignty with zero cloud dependency.
- Client
- Regional Public Sector Authority (NDA)
- Industry
- Government
- Type
- AI & Automation
- Year
- 2024
- Our role
- AI Lead & Backend Engineer
- Timeline
- 1 week
- Team
- 1 dev
Overview
A regional government authority needed to digitize thousands of physical archival records containing mixed English and Devanagari scripts.
Process.
- Designed a multi-stage pipeline: image enhancement via OpenCV → OCR text extraction → local LLM script filtering via Ollama → structured SQL storage → Streamlit dashboard for staff review.
- Each stage was optimized for accuracy and speed on government-spec hardware.
What we built.
- featCustom OpenCV image preprocessing (denoising, deskewing, thresholding)
- featMulti-engine OCR using Tesseract and EasyOCR for maximum accuracy
- featLocal LLM (Llama-3 via Ollama) for Devanagari/English script separation
- featFastAPI pipeline orchestrating end-to-end document processing
- featAutomated schema mapping from raw OCR to structured SQL database
- featStreamlit dashboard for upload, review, and verification by staff
Hard parts.
fixPoor legacy scan quality producing inaccurate OCR output
Built custom OpenCV pipeline with adaptive thresholding, morphological transformations, and contrast normalization, increasing OCR confidence scores by 35%.
fixDifferentiating Devanagari and English script in single documents
Prompt-engineered a local Llama-3 model via Ollama as a linguistic classifier, achieving high-speed and accurate script separation without external APIs.
fixStrict government data privacy requirements
Architected the entire system to run on-premise using local LLMs and local OCR engines, eliminating all cloud dependencies (OpenAI, Google Vision, etc.).
fixNon-technical staff needed to operate the system
Deployed a Streamlit UI allowing staff to upload scanned images, monitor processing, and review extracted database entries in real-time, no technical knowledge required.
Built with
Python OpenCV Ollama FastAPI Streamlit Tesseract SQL
Building something like this?
Tell us where the product is today. We reply within 12 hours.