# Late-arriving records

> Late-arriving records are facts that reach the pipeline after the period they belong to was already processed, so past totals need fixing.

Late-arriving records are facts that reach the pipeline after the period they belong to was already processed, such as an offline mobile event or a partner file delivered days late. If the pipeline ignores them, totals for past periods stay wrong.

For example, a delivery app records orders on drivers' phones and uploads them when the phone has a connection. An order placed on Monday evening in an area without coverage reaches the pipeline on Wednesday morning. By then, Monday's totals have been computed and published. To be correct, the pipeline must add the order to Monday, recompute Monday's aggregates and publish them again, while leaving Tuesday alone unless something else changed there.

Two timestamps make this work: the event time, when the order happened, and the arrival or load time, when the pipeline received it. Partitioning facts by event time, and tracking which event-time partitions received new rows in each load, tells the pipeline exactly which windows to rebuild. A limit on how late an event may be, such as seven days, keeps the reprocessing bounded; anything later is logged for review.

The dimension side has its own form of the problem: a fact can refer to a customer the pipeline has not seen yet. A common answer is to insert a placeholder dimension row, sometimes called an inferred member, and fill in its attributes when the customer record arrives, so the fact is not dropped and the join still works. See also [backfill](https://emolysis.com/glossary/backfill), [partitioning](https://emolysis.com/glossary/partitioning) and [incremental load](https://emolysis.com/glossary/incremental-load).

Platforms help with the mechanics: on Databricks, a [merge](https://emolysis.com/glossary/merge-upsert) into a Delta table can update the affected days in place, and Snowflake's streams show which rows arrived since the last run. The decision of how late is too late is a business one, and it should be written down next to the pipeline. See also [watermark](https://emolysis.com/glossary/watermark).

Handling them means reprocessing the affected windows and linking facts whose [dimension](https://emolysis.com/glossary/dimension) rows have not arrived yet to a placeholder. Emolysis's [Late-arriving](https://emolysis.com/shapes/late-arriving) shape does both, tested with a fact that arrives two days late. See also [fact table](https://emolysis.com/glossary/fact-table) and [watermark](https://emolysis.com/glossary/watermark).

---

Page: https://emolysis.com/glossary/late-arriving-data. Emolysis is a product of Bandpey GmbH, Munich, Germany. It is in private development, and nothing is released yet.