In private development
In development. DuckDB is the local engine and test judge, not a place your pipelines run in production. Nothing is released yet.
DuckDB
DuckDB, the local engine.
DuckDB is the engine Emolysis runs on your machine: where samples land, templates run and golden tests decide what may deploy. It is not a production target.
Local dev loop
Read
CSV, Parquet, JSON, Delta and Iceberg
Land
A local lakehouse per project
Prove
Golden tests compare actual and expected
Deploy
To your production platform
Overview
Two roles, one engine.
DuckDB is the engine Emolysis runs on your machine. It has two roles: the development loop, and the judge for golden tests. It is not a production target.
Each project keeps a local lakehouse in its folder. Samples you pull from your platform land there, and DuckDB reads CSV, Parquet and JSON files, and Delta and Iceberg tables, without an export step.
Every template's golden tests run here: a fixed input, an expected output and an exact comparison. The agent lints every pipeline it writes for your platform and runs its golden tests before anything is deployed. A red test sends the agent back to the template, and the agent deploys only after a green run.
Testing on a local sample spends no warehouse compute. Reading the sample from your platform is a query on your platform. Work proven on DuckDB is translated into your platform's dialect and deployed there.
Local profile
What the profile captures.
DuckDB uses the same profile format as the platforms, with fields of its own. This is what the local profile captures.
- Dialect
- DuckDB SQL
- Storage
- A local lakehouse for each project, kept in the project folder
- Reading
- CSV, Parquet and JSON files, and Delta and Iceberg tables, without an export step
- Role
- Where samples land, templates run and golden tests compare actual output with expected output
- Checks
- Lint for your platform, and golden tests on DuckDB, before any deploy
- Deploy target
- None. Work proven here deploys to your production platform
Setup
Nothing to connect.
DuckDB ships inside Emolysis, so you do not install a database separately.
Without a platform
Signed in to your Emolysis account, you can work on local files, CSV and Parquet, on DuckDB alone. That is enough to try templates, run golden tests and build a dashboard from a local file.
With a platform
Once you connect a platform, the samples you ask for are read into the project's local lakehouse, and the same golden tests run against them.
On your machine
Running templates and golden tests on DuckDB with files already on your machine needs no platform connection. Writing code needs the model provider you choose.
What leaves your machine
Tests run locally, but to write code the agent sends prompts to the AI model provider you choose. A prompt can include table and column names, schemas and sample values. With a paid plan's model usage, prompts pass through Emolysis's metering service, which is designed not to store or log their content.
Deploy
Proven here, deployed there.
DuckDB has no deploy target. Work proven on DuckDB is translated into your platform's SQL dialect and deployed there as its native unit.
Fixed input, expected output
A golden test runs a pipeline on fixed fixtures and compares the result with an expected output, row for row. If they differ, the test fails.
No cluster to wait for
DuckDB starts instantly and needs no cluster, so the tests can run on every save, not only before a release.
Then the real platform
A template is only called ready on a platform after its cases pass on that platform too, not only on DuckDB.
Dashboards from local files
A dashboard built from a local gold table draws on your machine. Publishing it as a shared link needs a paid plan.
Shapes
Ten golden tests.
Every shape's golden test runs on DuckDB before deploy. This is what each one checks.
- Ingest (batch)Loads three files, one of them malformed, and checks that the good files land exactly once and the bad one is reported instead of half-loaded.
- Incremental appendRuns the pipeline twice with overlapping input and checks that the second run adds only the new rows, with no duplicates and no gaps.
- CDC mergeFeeds inserts, updates and deletes, including several changes to one key in one batch, and compares the target with the expected final state.
- SCD Type 2Changes an attribute across three runs and checks for one current version per key, with validity ranges that neither overlap nor leave gaps.
- SnapshotTakes two snapshots and checks that both are kept with the right dates, that a re-run replaces instead of duplicating, and that retention removes only what it should.
- BackfillStarts from a target with a gap in the range and checks that the gap is filled, partitions outside the range are unchanged and a second run gives the same result.
- Late-arrivingDelivers a fact two days late and checks that the affected daily totals are corrected and nothing outside the window changes.
- Dedupe / conformFeeds duplicates and type drift and checks for one row per key and the right types, with drifted rows cast or quarantined as specified.
- Aggregation / goldUses input with known totals and checks every aggregate against them, including totals that cross a one-to-many join.
- Reverse ETLPushes the same batch twice to a stand-in destination and checks that the second push changes nothing.
All ten shapes, with their templates and golden tests, are on the pipeline shapes page.
Read next
Glossary
Terms on this page
Questions
DuckDB questions.
Answers about DuckDB and how it fits. More in the FAQ.
Why test on DuckDB instead of on my platform?
Speed and cost. DuckDB runs on your machine, starts instantly and needs no cluster, so a golden test spends no warehouse compute. That makes it practical to run the tests on every save, not only before a release. DuckDB is the judge for the expected outputs; the same transformation is then translated into your platform's dialect for deployment.
What is a golden test?
A golden test runs a pipeline on a fixed input and compares its output with an expected output, row for row. If they differ, the test fails. Each Emolysis template ships with golden tests built for its shape. The tests run on DuckDB on your machine. A red result sends the agent back to the template, and the agent deploys only after a green run.
Do I need to install DuckDB?
No. Each desktop build includes the engine and DuckDB, so you do not install a database separately. There is nothing to download yet.
What does local-first mean here?
It means the development loop runs on your machine: samples in DuckDB, templates filled and golden tests run locally, so iterating spends no warehouse compute. It does not mean that nothing leaves your machine. To write code, the agent sends prompts to the AI model provider you choose, and a prompt can include table and column names, schemas and sample values. With a paid plan's model usage, those prompts pass through Emolysis's metering service, which is designed not to store or log their content.
Can I use DuckDB without an account?
No. Every use of Emolysis needs an Emolysis account, local files included; the Free plan is one. Signed in, you can work on local files, CSV and Parquet, on DuckDB alone, with no platform connected. That is enough to try templates, run golden tests and build a dashboard from a local file.
Nothing is released yet
Prove it on your machine.
DuckDB ships inside Emolysis, so there is nothing to connect. Early access is by invitation, starting with data engineering teams on Databricks.