tag
4 items · 3 projects · 1 feed item
A multi-tenant data-quality platform: a Python SDK that runs inside each team's Databricks workspace, a FastAPI + Streamlit control plane, a fact/dim PostgreSQL store for results, threshold-based Slack alerting, and a no-code UI for non-engineers. Built on Great Expectations with a custom-expectations API for team-contributed rules.
A Pydantic-based data-validation library for the team that runs API analytics. Engineers point it at a JSON event, it generates a schema, they edit it to add field-level rules (null, range, allowed-values, custom), and the orchestrator runs validations every ingest. Distributed as an installable wheel.
Lead engineer on an enterprise data-quality platform with a pluggable connector framework for profiling, validation, and reconciliation across heterogeneous sources. Adopted across multiple internal customer engagements.
Architectural pattern for a multi-tenant data-quality platform across many teams. Why centralising contracts beats centralising data, and how the SDK + control plane + fact/dim store pattern works regardless of which tools you reach for.