Skip to content
Vol. I · No. 251
Mon · 8 Jun
A Daily Lexicon of Trustworthy Data
The Lexicon

005·75

backfill

/ˈbæk.fɪl/ - n.

1 [colloq.] Re-running history to match the present, and discovering history had a different opinion about what 'customer' meant.Keep. Punchy.This is the problem.

Working definition

2. Reprocessing historical data through a pipeline to populate or correct records for periods before a job existed or after it failed.

Filed
Seen in the wild · 2
See also
  • data pipelineA sequence of steps that worked when one person built it and has been load-bearing ever since.
  • idempotencyThe property everyone assumed the job had until the day it ran twice.
  • incremental loadProcessing only what changed, defined by a watermark everyone trusts and no one can explain.
  • replayLiving the same afternoon again, this time with the join written correctly.