← All work

Data Engineering · 2026

Warehouse Access Audit

Pattern

A recurring audit of table and view access across a cloud data warehouse, built so a platform team can see what downstream consumers actually depend on before changing or removing anything.

Problem

As a data platform evolves, it gets hard to say with confidence which tables and views are still in use. Without that picture, nothing can be safely deprecated, and consumers reading directly from raw layers they should not depend on go unnoticed.

Approach

  1. 01Wrote an access audit that reads warehouse query history to attribute reads to specific tables and views.
  2. 02Wired it into a scheduled Airflow DAG so the picture stays current instead of being rebuilt by hand.
  3. 03Added automated alerting for unused tables and risky direct-read patterns.
  4. 04Profiled the heavier scoring queries against the warehouse statement timeout and worked on bringing them under it.

Outcome

Cleanup and deprecation decisions now run on measured usage instead of guesswork. Incremental query optimization to widen coverage is ongoing.

Stack

  • Apache Airflow
  • SQL
  • Python
  • Cloud data warehouse