# Orchestration

> Orchestration runs pipeline steps in the right order, on a schedule or when data arrives, with retries, dependencies and alerts.

Orchestration runs pipeline steps in the right order, on a schedule or when data arrives, with retries, dependencies and alerts. Each platform has its own: Databricks jobs and pipelines, Snowflake tasks, and Fabric Data Pipelines.

For example, a nightly sales pipeline might have five tasks: load orders, load stores, build the silver orders table once both loads finish, build the gold monthly table after that, and refresh a dashboard last. An orchestrator runs the two loads in parallel at 02:00, starts each later task only when the tasks it depends on succeed, retries a failed load twice with a pause between tries, and alerts the engineer on call if the gold table is not ready by 06:00.

On Databricks, this is a job with tasks and dependencies, or a declarative pipeline whose tables are ordered by their references; on Snowflake, tasks chained into a graph, with streams so a task runs only when new data has arrived; on Microsoft Fabric, a Data Pipeline with activities. General-purpose orchestrators also exist, and some teams use one across several platforms, at the cost of one more system to run.

Good orchestration makes failure routine: each task is [idempotent](https://emolysis.com/glossary/idempotency), so a retry is safe; dependencies are explicit, so nothing reads half-built tables; and every run leaves a record of what ran, when and with which inputs. Triggering on data arrival rather than on a clock avoids wasted runs but needs care when a source delivers late. See also [lineage](https://emolysis.com/glossary/lineage) and [backfill](https://emolysis.com/glossary/backfill).

Platform-native scheduling also keeps the operational picture in one place: run history, logs, alerts and permissions sit next to the data the jobs write. See also [Databricks Asset Bundle](https://emolysis.com/glossary/asset-bundle) and [ELT](https://emolysis.com/glossary/elt).

Emolysis deploys pipelines in your platform's native units, so they run on the orchestration your platform already provides and your team can operate them with the tools it knows. See [Deploying](https://emolysis.com/docs/deploying) and [Databricks Asset Bundle](https://emolysis.com/glossary/asset-bundle). Retries and dependencies are easy to get wrong by hand, so they belong to the platform's own scheduler.

---

Page: https://emolysis.com/glossary/orchestration. Emolysis is a product of Bandpey GmbH, Munich, Germany. It is in private development, and nothing is released yet.