Orchestration runs pipeline steps in the right order, on a schedule or when data arrives, with retries, dependencies and alerts. Each platform has its own: Databricks jobs and pipelines, Snowflake tasks, and Fabric Data Pipelines.
For example, a nightly sales pipeline might have five tasks: load orders, load stores, build the silver orders table once both loads finish, build the gold monthly table after that, and refresh a dashboard last. An orchestrator runs the two loads in parallel at 02:00, starts each later task only when the tasks it depends on succeed, retries a failed load twice with a pause between tries, and alerts the engineer on call if the gold table is not ready by 06:00.
On Databricks, this is a job with tasks and dependencies, or a declarative pipeline whose tables are ordered by their references; on Snowflake, tasks chained into a graph, with streams so a task runs only when new data has arrived; on Microsoft Fabric, a Data Pipeline with activities. General-purpose orchestrators also exist, and some teams use one across several platforms, at the cost of one more system to run.
Good orchestration makes failure routine: each task is idempotent, so a retry is safe; dependencies are explicit, so nothing reads half-built tables; and every run leaves a record of what ran, when and with which inputs. Triggering on data arrival rather than on a clock avoids wasted runs but needs care when a source delivers late. See also lineage and backfill.
Platform-native scheduling also keeps the operational picture in one place: run history, logs, alerts and permissions sit next to the data the jobs write. See also Databricks Asset Bundle and ELT.
Emolysis deploys pipelines in your platform's native units, so they run on the orchestration your platform already provides and your team can operate them with the tools it knows. See Deploying and Databricks Asset Bundle. Retries and dependencies are easy to get wrong by hand, so they belong to the platform's own scheduler.