Skip to content
Vol. I · No. 251
Mon · 8 Jun
A Daily Lexicon of Trustworthy Data
The Lexicon

005·75

incremental load

/ˌɪn.krə.ˈmɛn.təl ləʊd/ - n.

1 [colloq.] Processing only what changed, defined by a watermark everyone trusts and no one can explain.Keep. Punchy.This is the problem.

Working definition

2. A load strategy that processes only new or changed records since the last run, rather than reloading the full dataset.

Filed
See also
  • backfillRe-running history to match the present, and discovering history had a different opinion about what 'customer' meant.
  • change data captureCapturing every change the source makes, including the ones it would prefer you had not noticed.
  • data martThe version of the numbers a team built so it would stop having to agree with the other teams.
  • watermarkA single stored number that is the difference between an incremental load and a four-hour incident.