005·75

data pipeline

/ˈdeɪ.tə ˈpaɪp.laɪn/ - n.

1 [colloq.] A sequence of steps that worked when one person built it and has been load-bearing ever since.Keep. Punchy.This is the problem.

Working definition

2. A defined sequence of steps that moves and transforms data from a source to a destination on a schedule.

Promoted

See also

brittle jobA job documented entirely in the muscle memory of the one person who knows which step to re-run first.
etlThree verbs, four teams, and no agreement on which one owns the part that broke.
orchestrationThe discipline of deciding which job runs first, now practiced by three tools that each believe they are in charge.
pipeline ownershipA field in the catalog set to the name of someone who left in March.