Data engineering
Muhammad
Omar Farooq
I build and run ELT on Snowflake, dbt, and Airflow — idempotent pipelines, clear models, and observability you can act on. When AI helps, it sits on data that is already trustworthy.
Selected proof
Selected work
Production delivery at work, and open-source tools I build on the side.
A CLI that turns YAML data contracts into SQL checks and an integrity scorecard — for verifying seams between systems.
$ pip install trustline $ trustline audit --demo
Community
Helped triage a
dbt-core
bug report: reproduced a
uv / CLI issue,
confirmed it matched a known Python 3.14 failure with a fix already in flight, and helped the reporter get unblocked.
Approach
Move data safely, model it clearly, and only add AI where the signal is already trustworthy.
Idempotent pipelines, cursor pagination, merges that stay dedup-safe under retry.
SCD Type-2, analytics marts, and schemas teams can trust in production.
MWAA with Git-based deploys — repeatable, reviewable, boring in the best way.
Freshness frameworks, dbt tests, versioned boundaries that fail loud on drift.
Snowflake procedures → Slack so incidents surface before stakeholders do.
Prior work included LLM retrieval and agents. I treat that as a layer on top of reliable pipelines — not a substitute for them.
Education
Graduate study in AI at Lahore University of Management Sciences (SBASSE) — deepening the theory behind production data and model systems.
Bachelor’s in Computer Engineering, University of Engineering and Technology, Taxila.
Publications
IEEE ICOSST 2024 · IEEE Xplore
Engineering Failure Analysis, Vol. 142, Article 106770 · ScienceDirect
Experience
Own Snowflake and dbt ELT across 5+ production systems, including bidirectional HubSpot ↔ Snowflake sync over 4M+ contacts. Observability, alerting, and CI/CD on MWAA.
ETL/ELT with Airflow, Airbyte, dbt, SQL, and Python. Dimensional models, SCD Type-2, automated validation.
LLM retrieval with LlamaIndex, MongoDB Atlas, and GPT-4; multi-agent workflows with CrewAI.