_Update, 2026-10-23: all thirty templates exist, one for each of the ten shapes on Databricks, Snowflake and Microsoft Fabric, and each passes its golden tests on DuckDB. None has run on a real platform yet, and none is released._
A general coding agent treats every request as new. Data engineering rarely is. Most pipelines in production are one of a small number of shapes, and each shape has a correct form that experienced engineers write the same way every time, with the same edge cases handled. Emolysis is being built around that observation. When you ask for a pipeline, the agent is designed to pick the shape first, and then fill a tested template for your platform.
The ten shapes
- Ingest (batch). Land files, extracts or database tables in bronze, typed, with load metadata, and with duplicates dropped on load.
- Incremental append. Append only the rows that arrived since the last run, tracked by a watermark that moves only after the load commits.
- CDC merge. Apply inserts, updates and deletes from a change feed, with the latest change per key winning.
- SCD Type 2. Keep the history of a dimension, with a validity range for every version and exactly one current row per key.
- Snapshot. Copy a source in full on a schedule and keep dated copies for a set retention period.
- Backfill. Re-run a pipeline for a past date range, replacing exactly those partitions.
- Late-arriving. Reprocess the right windows when facts arrive after their period was processed.
- Dedupe / conform. Turn bronze into silver: one row per key, with consistent types and names.
- Aggregation / gold. Build metrics tables at a declared grain for dashboards and reports.
- Reverse ETL. Push gold data back into business applications, safe to re-run.
Why shapes, not free-form code
Each shape fails in known ways. A merge picks an arbitrary row when a key appears twice in one batch. An SCD Type 2 load run twice creates duplicate versions. An incremental load with the wrong boundary drops the rows at the edge. A backfill appends where it should replace and double counts a month. These are not exotic bugs. They are the bugs that reach production because the code looked right.
A template written for the shape handles those cases once, and its golden tests check them. For each shape and each platform, Emolysis will have a template folder: a SQL or Python file with typed slots, test fixtures built around the shape's hard cases, expected outputs, a cost note and the platform features it needs. Ten shapes on three platforms will make thirty templates.
The agent's job will then become narrower and easier to check. It will decide which shape fits, fill the slots from your catalog and your request, and ask you only for what is missing, such as which column is the business key. You will review a filled template, not a blank page. The templates and slots docs describe the format.
Shapes combine
Real pipelines are chains. A customer pipeline might land a source with Ingest (batch), clean it with Dedupe / conform, keep history with SCD Type 2, and feed revenue metrics through Aggregation / gold. Emolysis is designed to build each step from its own template, with its own tests, and connect them.
When nothing fits
Some work will not match any shape. When that happens, the agent is designed to say so rather than improvise something it cannot test. You will be able to write that part yourself in the same project, with your own golden tests. The template format is public, so you will also be able to turn your own recurring patterns into templates for your projects. See what if my pipeline does not fit a shape.
How we will measure ourselves
Every cell of the shape-by-platform matrix will be marked green, yellow or missing. Yellow means the template passes its golden tests on DuckDB; green means it also passes on the real platform. We are writing the Databricks templates first. None are released yet, and when we publish the matrix, it will show the missing cells too. The docs on shapes explain how the agent will pick one.
Emolysis is in private development. Nothing described here is released yet.