Databricks
The Databricks template uses window functions with a total order, QUALIFY in Databricks SQL and Delta constraints for checks.
- ROW_NUMBER with QUALIFY keeps one row per key in a single statement. Add every ordering column and a final tie-breaker so the result is the same on every run.
- Delta CHECK and NOT NULL constraints fail a write that violates them. The template uses them where a hard stop is right and a quarantine table where it is not.
- Whether a failed cast raises an error or returns null depends on the compute and its ANSI setting, so the template uses try_cast explicitly and checks for the nulls it produces.
- Silver tables live in Unity Catalog, so the schema's grants decide who may read the cleaned data and who may read the quarantine table.