Mustashar Data Platform
An end-to-end platform that collects education regulations, extracts Arabic text, builds document knowledge graphs, and brings pipeline operations into one live admin workspace.
Inside the Platform
Screenshots from the running application. Open any image to inspect it at full size.
Timeline
Built iteratively across source collection, OCR, knowledge graph extraction, orchestration, and the admin monitor.
Role & Scope
Developed the Python pipelines, storage integrations, job orchestration, and the Arabic Next.js monitoring and document exploration interface.
Team Setup
End-to-end implementation spanning data engineering, AI processing, and full-stack development.
Context
Problem: Regulations, rules, ministry systems, and salary tables arrive in different formats. Collecting them is only the first step: extraction, provenance, storage, and failed processing stages also need to be traceable.
Solution: Built four source pipelines with scheduled execution, resumable stages, Arabic OCR, and PostgreSQL/Neo4j loading, alongside a Next.js monitor for inspecting documents, tracking costs, and managing operations.
Impact: Connected collection, extraction, and exploration in one workspace: operators can follow processing, resume unfinished stages, inspect source PDFs and extracted text, and explore document-scoped knowledge graphs.
Implementation Details
Architecture
Docker Compose runs a scheduler, worker, Next.js monitor, pipeline PostgreSQL, a separate monitoring PostgreSQL, Neo4j, and MinIO. PDF jobs follow preflight, scrape, OCR, graph build, load, and validation; systems and salary sources use structured extraction paths.
Data & Evaluation
Final validation checks loaded records against staged data. The document workspace exposes original PDFs, extracted Markdown, chunks, entities, and evidence so processing outputs can be inspected against their sources.
Production & Ops
Checkpoint-based retries reuse completed work. The monitor provides live logs, service health, per-model token and estimated cost tracking, model fallback settings, weekly schedule controls, role-based accounts, and an administration audit log.
Lessons Learned
Reliable document processing needs visibility at every stage: retaining source evidence, recording model usage immediately, and resuming unfinished work make extraction easier to operate and debug.
Links
Last verified: September 1, 2026
Public Evidence
Last verified: September 1, 2026
This case study summarizes production work delivered under private repositories and confidentiality constraints. Public artifacts are intentionally omitted while keeping the implementation narrative transparent.